index.src.html

<pre class='metadata'>
Title: WebCodecs
Repository: w3c/webcodecs
Status: ED
ED: https://w3c.github.io/webcodecs/
TR: https://www.w3.org/TR/webcodecs/
Shortname: webcodecs
Level: None
Group: mediawg
Editor: Paul Adenot, w3cid 62410, Mozilla https://www.mozilla.org/
Editor: Eugene Zemtsov, w3cid 139375, Google LLC https://www.google.com/
Former Editor: Bernard Aboba, w3cid 65611, Microsoft Corporation https://www.microsoft.com/
Former Editor: Chris Cunningham, w3cid 114832, Google Inc. https://www.google.com/

Abstract: This specification defines interfaces to codecs for encoding and
    decoding of audio, video, and images.

    This specification does not specify or require any particular codec or
    method of encoding or decoding. The purpose of this specification is to
    provide JavaScript interfaces to implementations of existing codec
    technology developed elsewhere. Implementers are free to
    support any combination of codecs or none at all.

Complain About:accidental-2119 yes
Markup Shorthands:css no, markdown yes, dfn yes
!Participate: <a href="https://github.com/w3c/webcodecs">Git Repository.</a>
!Participate: <a href="https://github.com/w3c/webcodecs/issues/new">File an issue.</a>
!Version History: <a href="https://github.com/w3c/webcodecs/commits">https://github.com/w3c/webcodecs/commits</a>
</pre>

<pre class=link-defaults>
spec:html; type:attribute; text:hidden
spec:html; type:enum-value; text:srgb
spec:infra; type:dfn; text:list
spec:infra; type:dfn; text:enqueue
spec:webaudio-1.1; type:interface; text:AudioWorklet
spec:webaudio-1.1; type:interface; text:AudioBuffer
</pre>

<pre class='anchors'>
spec: html; urlPrefix: https://html.spec.whatwg.org/multipage/;
    for: ImageBitmap;
        type: dfn; text: bitmap data; url: imagebitmap-and-animations.html#concept-imagebitmap-bitmap-data
    for: ImageData
        type: dfn; text: canvas pixel data; url: canvas.html#imagedata
    for: Canvas;
        type: dfn; text: Check the usability of the image argument; url: canvas.html#check-the-usability-of-the-image-argument
        type: dfn; text: is not origin-clean; url: canvas.html#the-image-argument-is-not-origin-clean
    for: webappapis;
        type: dfn; text: entry settings object; url: webappapis.html#entry-settings-object
    for: media;
        type: dfn; text: current playback position; url: media.html#current-playback-position
    type: dfn; text: live; url: infrastructure.html#live

spec: webrtc-svc; urlPrefix: https://www.w3.org/TR/webrtc-svc/
    type: dfn; text: scalability mode identifier; url:#scalabilitymodes*

spec: mst-content-hint; urlPrefix: https://www.w3.org/TR/mst-content-hint/
    type: dfn; text: video content hints; url:#video-content-hints

spec: ECMASCRIPT; urlPrefix: https://tc39.es/ecma262/
    type: dfn; text: the current Realm; url: #current-realm
</pre>

<pre class='biblio'>
{
  "H.273": {
    "href": "https://www.itu.int/rec/T-REC-H.273/en",
    "title": "Coding-independent code points for video signal type identification",
    "publisher": "ITU",
    "date": "December 2016"
  },
  "WEBCODECS-VIDEO-FRAME-METADATA-REGISTRY": {
    "href": "https://w3c.github.io/webcodecs/video_frame_metadata_registry.html",
    "title": "WebCodecs VideoFrame Metadata Registry",
    "publisher": "W3C",
    "authors": [
      "Youenn Fablet"
    ],
    "status": "ED"
  }
}
</pre>

<style>
main > dl > dd {
  margin-bottom: 1em;
}

table {
  width: 100%;
}

table#sample-types td, table#sample-types th {
  text-align: center;
}

table#sample-types .even {
    background-color: lightgrey;
}

.rfc2119 {
  text-transform: lowercase;
  font-variant: small-caps;
  font-style: normal;
}

</style>


Definitions {#definitions}
==========================

: <dfn>Codec</dfn>
:: Refers generically to an instance of {{AudioDecoder}}, {{AudioEncoder}},
    {{VideoDecoder}}, or {{VideoEncoder}}.

: <dfn lt="Key Chunk|Key Frame">Key Chunk</dfn>
:: An encoded chunk that does not depend on any other frames for decoding. Also
    commonly referred to as a "key frame".

: <dfn>Internal Pending Output</dfn>
:: Codec outputs such as {{VideoFrame}}s that currently reside in the internal
    pipeline of the underlying codec implementation. The underlying codec
    implementation <em class="rfc2119">MAY</em> emit new outputs only when new
    inputs are provided. The underlying codec implementation <em
    class="rfc2119">MUST</em> emit all outputs in response to a flush.

: <dfn lt="system resources">Codec System Resources</dfn>
:: Resources including CPU memory, GPU memory, and exclusive handles to specific
    decoding/encoding hardware that <em class="rfc2119">MAY</em> be allocated by
    the User Agent as part of codec configuration or generation of {{AudioData}}
    and {{VideoFrame}} objects. Such resources <em class="rfc2119">MAY</em> be
    quickly exhausted and <em class="rfc2119">SHOULD</em> be released immediately
    when no longer in use.

: <dfn>Temporal Layer</dfn>
:: A grouping of {{EncodedVideoChunk}}s whose timestamp cadence produces a
    particular framerate. See {{VideoEncoderConfig/scalabilityMode}}.

: <dfn>Progressive Image</dfn>
:: An image that supports decoding to multiple levels of detail, with lower
    levels becoming available while the encoded data is not yet fully buffered.

: <dfn>Progressive Image Frame Generation</dfn>
:: A generational identifier for a given [=Progressive Image=] decoded output.
    Each successive generation adds additional detail to the decoded output.
    The mechanism for computing a frame's generation is implementer defined.

: <dfn>Primary Image Track</dfn>
:: An image track that is marked by the given image file as being the default
    track. The mechanism for indicating a primary track is format defined.

: <dfn>RGB Format</dfn>
:: A {{VideoPixelFormat}} containing red, green, and blue color channels in
    any order or layout (interleaved or planar), and irrespective of whether an
    alpha channel is present.

: <dfn>sRGB Color Space</dfn>
:: A {{VideoColorSpace}} object, initialized as follows:
    1. {{VideoColorSpace/[[primaries]]}} is set to {{VideoColorPrimaries/bt709}},
    2. {{VideoColorSpace/[[transfer]]}} is set to {{VideoTransferCharacteristics/iec61966-2-1}},
    3. {{VideoColorSpace/[[matrix]]}} is set to {{VideoMatrixCoefficients/rgb}},
    4. {{VideoColorSpace/[[full range]]}} is set to `true`

: <dfn>Display P3 Color Space</dfn>
:: A {{VideoColorSpace}} object, initialized as follows:
    1. {{VideoColorSpace/[[primaries]]}} is set to {{VideoColorPrimaries/smpte432}},
    2. {{VideoColorSpace/[[transfer]]}} is set to {{VideoTransferCharacteristics/iec61966-2-1}},
    3. {{VideoColorSpace/[[matrix]]}} is set to {{VideoMatrixCoefficients/rgb}},
    4. {{VideoColorSpace/[[full range]]}} is set to `true`

: <dfn>REC709 Color Space</dfn>
:: A {{VideoColorSpace}} object, initialized as follows:
    1. {{VideoColorSpace/[[primaries]]}} is set to {{VideoColorPrimaries/bt709}},
    2. {{VideoColorSpace/[[transfer]]}} is set to {{VideoTransferCharacteristics/bt709}},
    3. {{VideoColorSpace/[[matrix]]}} is set to {{VideoMatrixCoefficients/bt709}},
    4. {{VideoColorSpace/[[full range]]}} is set to `false`

: <dfn lt="saturated">Codec Saturation</dfn>
:: The state of an underlying codec implementation where the number of active
    decoding or encoding requests has reached an implementation specific
    maximum such that it is temporarily unable to accept more work. The maximum
    may be any value greater than 1, including infinity (no maximum). While
    saturated, additional calls to `decode()` or `encode()` will be buffered
    in the [=control message queue=], and will increment the respective
    `decodeQueueSize` and `encodeQueueSize` attributes. The codec implementation
    will become unsaturated after making sufficient progress on the current
    workload.

<dfn>Codec Processing Model</dfn> {#codec-processing-model-section}
===================================================================

Background {#processing-model-background}
-----------------------------------------

<div class=non-normative>
  This section is non-normative.

  The codec interfaces defined by the specification are designed such that new
  codec tasks can be scheduled while previous tasks are still pending. For
  example, web authors can call `decode()` without waiting for a previous
  `decode()` to complete. This is achieved by offloading underlying codec tasks
  to a separate [=parallel queue=] for parallel execution.

  This section describes threading behaviors as they are visible from the
  perspective of web authors. Implementers can choose to use more threads, as
  long as the externally visible behaviors of blocking and sequencing are
  maintained as follows.
</div>

Control Messages {#control-messages}
------------------------------------

A <dfn>control message</dfn> defines a sequence of steps corresponding to a
method invocation on a [=codec=] instance (e.g. `encode()`).

A <dfn>control message queue</dfn> is a [=queue=] of
[=control messages=]. Each [=codec=] instance has a control message queue
stored in an internal slot named
<dfn id='control-message-queue-slot'>[[control message queue]]</dfn>.

<dfn lt="Enqueues a control message|Queue a control message">Queuing a control
message</dfn> means [=enqueuing=] the message to a [=codec=]’s [=[[control
message queue]]=]. Invoking codec methods will generally queue a control message
to schedule work.

<dfn lt="running a control message|control message steps">Running a control
message</dfn> means performing a sequence of steps specified by the method
that enqueued the message.

The steps of a given control message can block processing later messages in the
control message queue. Each [=codec=] instance has a boolean internal slot named
<dfn>[[message queue blocked]]</dfn> that is set to `true` when this occurs. A
blocking message will conclude by setting [=[[message queue blocked]]=] to
`false` and rerunning the [=Process the control message queue=] steps.

All control messages will return either `"processed"` or `"not processed"`.
Returning `"processed"` indicates the message steps are being (or have been)
executed and the message may be removed from the [=control message queue=].
`"not processed"` indicates the message must not be processed at this time
and should remain in the [=control message queue=] to be retried later.

To <dfn>Process the control message queue</dfn>, run these steps:
1. While [=[[message queue blocked]]=] is `false` and
    [=[[control message queue]]=] is not empty:
    1. Let |front message| be the first message in
        [=[[control message queue]]=].
    2. Let |outcome| be the result of running the [=control message steps=]
        described by |front message|.
    3. If |outcome| equals `"not processed"`, break.
    4. Otherwise, dequeue |front message| from the
        [=[[control message queue]]=].

Codec Work Parallel Queue {#codec-work-parallel-queue}
------------------------------------------------------

Each [=codec=] instance has an internal slot named
<dfn>[[codec work queue]]</dfn> that is a [=parallel queue=].

Each [=codec=] instance has an internal slot named
<dfn>[[codec implementation]]</dfn> that refers to the underlying platform
encoder or decoder. Except for the initial assignment, any steps that reference
[=[[codec implementation]]=] will be enqueued to the [=[[codec work queue]]=].

Each [=codec=] instance has a unique <dfn>codec task source</dfn>. Tasks
[=queue a task|queued=] from the [=[[codec work queue]]=] to the [=/event loop=]
will use the [=codec task source=].


AudioDecoder Interface {#audiodecoder-interface}
================================================

<xmp class='idl'>
[Exposed=(Window,DedicatedWorker), SecureContext]
interface AudioDecoder : EventTarget {
  constructor(AudioDecoderInit init);

  readonly attribute CodecState state;
  readonly attribute unsigned long decodeQueueSize;
  attribute EventHandler ondequeue;

  undefined configure(AudioDecoderConfig config);
  undefined decode(EncodedAudioChunk chunk);
  Promise<undefined> flush();
  undefined reset();
  undefined close();

  static Promise<AudioDecoderSupport> isConfigSupported(AudioDecoderConfig config);
};

dictionary AudioDecoderInit {
  required AudioDataOutputCallback output;
  required WebCodecsErrorCallback error;
};

callback AudioDataOutputCallback = undefined(AudioData output);
</xmp>

Internal Slots {#audiodecoder-internal-slots}
---------------------------------------------
: <dfn attribute for=AudioDecoder>[[control message queue]]</dfn>
:: A [=queue=] of [=control messages=] to be performed upon this [=codec=]
    instance. See [=[[control message queue]]=].
: <dfn attribute for=AudioDecoder>[[message queue blocked]]</dfn>
:: A boolean indicating when processing the
    {{AudioDecoder/[[control message queue]]}} is blocked by a pending
    [=control message=]. See [=[[message queue blocked]]=].
: <dfn attribute for=AudioDecoder>[[codec implementation]]</dfn>
:: Underlying decoder implementation provided by the User Agent. See
    [=[[codec implementation]]=].
: <dfn attribute for=AudioDecoder>[[codec work queue]]</dfn>
:: A [=parallel queue=] used for running parallel steps that reference the
    {{AudioDecoder/[[codec implementation]]}}. See [=[[codec work queue]]=].
: <dfn attribute for=AudioDecoder>[[codec saturated]]</dfn>
:: A boolean indicating when the {{AudioDecoder/[[codec implementation]]}} is
    unable to accept additional decoding work.
: <dfn attribute for=AudioDecoder>[[output callback]]</dfn>
:: Callback given at construction for decoded outputs.
: <dfn attribute for=AudioDecoder>[[error callback]]</dfn>
:: Callback given at construction for decode errors.
: <dfn attribute for=AudioDecoder>[[key chunk required]]</dfn>
:: A boolean indicating that the next chunk passed to {{AudioDecoder/decode()}}
    <em class="rfc2119">MUST</em> describe a [=key chunk=] as indicated by
    {{EncodedAudioChunk/[[type]]}}.
: <dfn attribute for=AudioDecoder>\[[state]]</dfn>
:: The current {{CodecState}} of this {{AudioDecoder}}.
: <dfn attribute for=AudioDecoder>\[[decodeQueueSize]]</dfn>
:: The number of pending decode requests. This number will decrease as the
    underlying codec is ready to accept new input.
: <dfn attribute for=AudioDecoder>[[pending flush promises]]</dfn>
:: A list of unresolved promises returned by calls to {{AudioDecoder/flush()}}.
: <dfn attribute for=AudioDecoder>[[dequeue event scheduled]]</dfn>
:: A boolean indicating whether a {{AudioDecoder/dequeue}} event is already
    scheduled to fire. Used to avoid event spam.

Constructors {#audiodecoder-constructors}
-----------------------------------------
<dfn constructor for=AudioDecoder title="AudioDecoder(init)">
  AudioDecoder(init)
</dfn>
1. Let d be a new {{AudioDecoder}} object.
2. Assign a new [=queue=] to {{AudioDecoder/[[control message queue]]}}.
3. Assign `false` to {{AudioDecoder/[[message queue blocked]]}}.
4. Assign `null` to {{AudioDecoder/[[codec implementation]]}}.
5. Assign the result of starting a new [=parallel queue=] to
    {{AudioDecoder/[[codec work queue]]}}.
6. Assign `false` to {{AudioDecoder/[[codec saturated]]}}.
7. Assign init.output to {{AudioDecoder/[[output callback]]}}.
8. Assign init.error to {{AudioDecoder/[[error callback]]}}.
9. Assign `true` to {{AudioDecoder/[[key chunk required]]}}.
10. Assign `"unconfigured"` to {{AudioDecoder/[[state]]}}
11. Assign `0` to {{AudioDecoder/[[decodeQueueSize]]}}.
12. Assign a new [=list=] to {{AudioDecoder/[[pending flush promises]]}}.
13. Assign `false` to {{AudioDecoder/[[dequeue event scheduled]]}}.
13. Return d.

Attributes {#audiodecoder-attributes}
-------------------------------------
: <dfn attribute for=AudioDecoder>state</dfn>
:: Returns the value of {{AudioDecoder/[[state]]}}.
: <dfn attribute for=AudioDecoder>decodeQueueSize</dfn>
:: Returns the value of {{AudioDecoder/[[decodeQueueSize]]}}.
: <dfn attribute for=AudioDecoder>ondequeue</dfn>
:: An [=event handler IDL attribute=] whose [=event handler event type=] is
    {{AudioDecoder/dequeue}}.

Event Summary {#audiodecoder-event-summary}
-------------------------------------------
: <dfn event for=AudioDecoder>dequeue</dfn>
:: Fired at the {{AudioDecoder}} when the {{AudioDecoder/decodeQueueSize}} has
    decreased.


Methods {#audiodecoder-methods}
-------------------------------
<dl>
  <dt><dfn method for=AudioDecoder>configure(config)</dfn></dt>
  <dd>
    [=Enqueues a control message=] to configure the audio decoder for decoding
    chunks as described by |config|.

    NOTE: This method will trigger a {{NotSupportedError}} if the User Agent
        does not support |config|. Authors are encouraged to first check support
        by calling {{AudioDecoder/isConfigSupported()}} with |config|. User
        Agents don't have to support any particular codec type or configuration.

    When invoked, run these steps:
    1. If |config| is not a [=valid AudioDecoderConfig=], throw a
        {{TypeError}}.
    2. If {{AudioDecoder/[[state]]}} is `“closed”`, throw an {{InvalidStateError}}.
    3. Set {{AudioDecoder/[[state]]}} to `"configured"`.
    4. Set {{AudioDecoder/[[key chunk required]]}} to `true`.
    5. [=Queue a control message=] to configure the decoder with |config|.
    6. [=Process the control message queue=].

    [=Running a control message=] to configure the decoder means running
    these steps:
    1. Assign `true` to {{AudioDecoder/[[message queue blocked]]}}.
    2. Enqueue the following steps to {{AudioDecoder/[[codec work queue]]}}:
        1. Let |supported| be the result of running the <a>Check Configuration
            Support</a> algorithm with |config|.
        2. If |supported| is `false`, [=queue a task=] to run the <a>Close
            AudioDecoder</a> algorithm with {{NotSupportedError}} and abort
            these steps.
        3. If needed, assign {{AudioDecoder/[[codec implementation]]}} with an
            implementation supporting |config|.
        4. Configure {{AudioDecoder/[[codec implementation]]}} with |config|.
        5. [=queue a task=] to run the following steps:
            1. Assign `false` to {{AudioDecoder/[[message queue blocked]]}}.
            2. [=Queue a task=] to [=Process the control message queue=].
    3. Return `"processed"`.
  </dd>

  <dt><dfn method for=AudioDecoder>decode(chunk)</dfn></dt>
  <dd>
    [=Enqueues a control message=] to decode the given |chunk|.

    When invoked, run these steps:
    1. If {{AudioDecoder/[[state]]}} is not `"configured"`, throw an
        {{InvalidStateError}}.
    2. If {{AudioDecoder/[[key chunk required]]}} is `true`:
        1. If |chunk|.{{EncodedAudioChunk/[[type]]}} is not
            {{EncodedAudioChunkType/key}}, throw a {{DataError}}.
        2. Implementers <em class="rfc2119">SHOULD</em> inspect the |chunk|'s
            {{EncodedAudioChunk/[[internal data]]}} to verify that
            it is truly a [=key chunk=]. If a mismatch is detected, throw a
            {{DataError}}.
        3. Otherwise, assign `false` to
            {{AudioDecoder/[[key chunk required]]}}.
    3. Increment {{AudioDecoder/[[decodeQueueSize]]}}.
    4. [=Queue a control message=] to decode the |chunk|.
    5. [=Process the control message queue=].

    [=Running a control message=] to decode the chunk means performing these
    steps:
    1. If {{AudioDecoder/[[codec saturated]]}} equals `true`, return `"not
        processed"`.
    2. If decoding chunk will cause the
        {{AudioDecoder/[[codec implementation]]}} to become [=saturated=],
        assign `true` to {{AudioDecoder/[[codec saturated]]}}.
    3. Decrement {{AudioDecoder/[[decodeQueueSize]]}} and run the
        [=AudioDecoder/Schedule Dequeue Event=] algorithm.
    4. Enqueue the following steps to the {{AudioDecoder/[[codec work queue]]}}:
        1. Attempt to use {{AudioDecoder/[[codec implementation]]}} to decode
            the chunk.
        2. If decoding results in an error, [=queue a task=] to run the
            [=Close AudioDecoder=] algorithm with {{EncodingError}} and return.
        3. If {{AudioDecoder/[[codec saturated]]}} equals `true` and
            {{AudioDecoder/[[codec implementation]]}} is no longer
            [=saturated=], [=queue a task=] to perform the following steps:
                1. Assign `false` to {{AudioDecoder/[[codec saturated]]}}.
                2. [=Process the control message queue=].
        4. Let |decoded outputs| be a [=list=] of decoded audio data outputs
            emitted by {{AudioDecoder/[[codec implementation]]}}.
        5. If |decoded outputs| is not empty, [=queue a task=] to run the
            [=Output AudioData=] algorithm with |decoded outputs|.
    5. Return `"processed"`.
  </dd>

  <dt><dfn method for=AudioDecoder>flush()</dfn></dt>
  <dd>
    Completes all [=control messages=] in the [=control message queue=]
    and emits all outputs.

    When invoked, run these steps:
    1. If {{AudioDecoder/[[state]]}} is not `"configured"`, return
        [=a promise rejected with=] {{InvalidStateError}} {{DOMException}}.
    2. Set {{AudioDecoder/[[key chunk required]]}} to `true`.
    3. Let |promise| be a new Promise.
    4. Append |promise| to {{AudioDecoder/[[pending flush promises]]}}.
    5. [=Queue a control message=] to flush the codec with |promise|.
    6. [=Process the control message queue=].
    7. Return |promise|.

    [=Running a control message=] to flush the codec means performing these
    steps with |promise|.
    1. Enqueue the following steps to the {{AudioDecoder/[[codec work queue]]}}:
        1. Signal {{AudioDecoder/[[codec implementation]]}} to emit all
            [=internal pending outputs=].
        2. Let |decoded outputs| be a [=list=] of decoded audio data outputs
            emitted by {{AudioDecoder/[[codec implementation]]}}.
        3. [=Queue a task=] to perform these steps:
            1. If |decoded outputs| is not empty, run the [=Output AudioData=]
                algorithm with |decoded outputs|.
            2. Remove |promise| from
                {{AudioDecoder/[[pending flush promises]]}}.
            3. Resolve |promise|.
    2. Return `"processed"`.
  </dd>

  <dt><dfn method for=AudioDecoder>reset()</dfn></dt>
  <dd>
    Immediately resets all state including configuration,
    [=control messages=] in the [=control message queue=], and all pending
    callbacks.

    When invoked, run the [=Reset AudioDecoder=] algorithm with an
    {{AbortError}} {{DOMException}}.
  </dd>

  <dt><dfn method for=AudioDecoder>close()</dfn></dt>
  <dd>
    Immediately aborts all pending work and releases [=system resources=].
    Close is final.

    When invoked, run the [=Close AudioDecoder=] algorithm with an
    {{AbortError}} {{DOMException}}.
  </dd>

  <dt><dfn method for=AudioDecoder>isConfigSupported(config)</dfn></dt>
  <dd>
    Returns a promise indicating whether the provided |config| is supported by
    the User Agent.

    NOTE: The returned {{AudioDecoderSupport}} {{AudioDecoderSupport/config}}
        will contain only the dictionary members that User Agent recognized.
        Unrecognized dictionary members will be ignored. Authors can detect
        unrecognized dictionary members by comparing
        {{AudioDecoderSupport/config}} to their provided |config|.

    When invoked, run these steps:
    1. If |config| is not a <a>valid AudioDecoderConfig</a>, return
        [=a promise rejected with=] {{TypeError}}.
    2. Let |p| be a new Promise.
    3. Let |checkSupportQueue| be the result of starting a new <a>parallel
        queue</a>.
    4. Enqueue the following steps to |checkSupportQueue|:
        1. Let |supported| be the result of running
            the <a>Check Configuration Support</a> algorithm with |config|.
        2. [=Queue a task=] to run the following steps:
            1. Let |decoderSupport| be a newly constructed
                {{AudioDecoderSupport}}, initialized as follows:
                1. Set {{AudioDecoderSupport/config}} to the result of running the
                    <a>Clone Configuration</a> algorithm with |config|.
                2. Set {{AudioDecoderSupport/supported}} to |supported|.
            2. Resolve |p| with |decoderSupport|.
    5. Return |p|.
  </dd>
</dl>

Algorithms {#audiodecoder-algorithms}
-------------------------------------
<dl>
  <dt><dfn for=AudioDecoder>Schedule Dequeue Event</dfn>
  <dd>
    1. If {{AudioDecoder/[[dequeue event scheduled]]}} equals `true`, return.
    2. Assign `true` to {{AudioDecoder/[[dequeue event scheduled]]}}.
    3. [=Queue a task=] to run the following steps:
        1. Fire a simple event named {{AudioDecoder/dequeue}} at [=this=].
        2. Assign `false` to {{AudioDecoder/[[dequeue event scheduled]]}}.
  </dd>
  <dt><dfn>Output AudioData</dfn> (with |outputs|)</dt>
  <dd>
    Run these steps:
    1. For each |output| in |outputs|:
        1. Let |data| be an {{AudioData}}, initialized as follows:
            1. Assign `false` to {{platform object/[[Detached]]}}.
            2. Let |resource| be the [=media resource=] described by |output|.
            3. Let |resourceReference| be a reference to |resource|.
            4. Assign |resourceReference| to
                {{AudioData/[[resource reference]]}}.
            5. Let |timestamp| be the {{EncodedAudioChunk/[[timestamp]]}} of the
                {{EncodedAudioChunk}} associated with |output|.
            6. Assign |timestamp| to {{AudioData/[[timestamp]]}}.
            7. If |output| uses a recognized {{AudioSampleFormat}}, assign that
                format to {{AudioData/[[format]]}}. Otherwise, assign `null` to
                {{AudioData/[[format]]}}.
            8. Assign values to {{AudioData/[[sample rate]]}},
                {{AudioData/[[number of frames]]}}, and
                {{AudioData/[[number of channels]]}} as determined by |output|.
        3. Invoke {{AudioDecoder/[[output callback]]}} with |data|.
  </dd>
  <dt><dfn>Reset AudioDecoder</dfn> (with |exception|)</dt>
  <dd>
    Run these steps:
    1. If {{AudioDecoder/[[state]]}} is `"closed"`, throw an {{InvalidStateError}}.
    2. Set {{AudioDecoder/[[state]]}} to `"unconfigured"`.
    3. Signal {{AudioDecoder/[[codec implementation]]}} to cease producing
        output for the previous configuration.
    4. Remove all [=control messages=] from the
        {{AudioDecoder/[[control message queue]]}}.
    5. If {{AudioDecoder/[[decodeQueueSize]]}} is greater than zero:
        1. Set {{AudioDecoder/[[decodeQueueSize]]}} to zero.
        2. Run the [=AudioDecoder/Schedule Dequeue Event=] algorithm.
    6. For each |promise| in {{AudioDecoder/[[pending flush promises]]}}:
        1. Reject |promise| with |exception|.
        2. Remove |promise| from {{AudioDecoder/[[pending flush promises]]}}.
  </dd>
  <dt><dfn>Close AudioDecoder</dfn> (with |exception|)</dt>
  <dd>
    Run these steps:
    1. Run the [=Reset AudioDecoder=] algorithm with |exception|.
    2. Set {{AudioDecoder/[[state]]}} to `"closed"`.
    3. Clear {{AudioDecoder/[[codec implementation]]}} and release associated
        [=system resources=].
    4. If |exception| is not an {{AbortError}} {{DOMException}},
        invoke the {{AudioDecoder/[[error callback]]}} with |exception|.
  </dd>
</dl>

VideoDecoder Interface {#videodecoder-interface}
================================================

<xmp class='idl'>
[Exposed=(Window,DedicatedWorker), SecureContext]
interface VideoDecoder : EventTarget {
  constructor(VideoDecoderInit init);

  readonly attribute CodecState state;
  readonly attribute unsigned long decodeQueueSize;
  attribute EventHandler ondequeue;

  undefined configure(VideoDecoderConfig config);
  undefined decode(EncodedVideoChunk chunk);
  Promise<undefined> flush();
  undefined reset();
  undefined close();

  static Promise<VideoDecoderSupport> isConfigSupported(VideoDecoderConfig config);
};

dictionary VideoDecoderInit {
  required VideoFrameOutputCallback output;
  required WebCodecsErrorCallback error;
};

callback VideoFrameOutputCallback = undefined(VideoFrame output);
</xmp>

Internal Slots {#videodecoder-internal-slots}
---------------------------------------------
: <dfn attribute for=VideoDecoder>[[control message queue]]</dfn>
:: A [=queue=] of [=control messages=] to be performed upon this [=codec=]
    instance. See [=[[control message queue]]=].
: <dfn attribute for=VideoDecoder>[[message queue blocked]]</dfn>
:: A boolean indicating when processing the
    {{VideoDecoder/[[control message queue]]}} is blocked by a pending
    [=control message=]. See [=[[message queue blocked]]=].
: <dfn attribute for=VideoDecoder>[[codec implementation]]</dfn>
:: Underlying decoder implementation provided by the User Agent. See
    [=[[codec implementation]]=].
: <dfn attribute for=VideoDecoder>[[codec work queue]]</dfn>
:: A [=parallel queue=] used for running parallel steps that reference the
    {{VideoDecoder/[[codec implementation]]}}. See [=[[codec work queue]]=].
: <dfn attribute for=VideoDecoder>[[codec saturated]]</dfn>
:: A boolean indicating when the {{VideoDecoder/[[codec implementation]]}} is
    unable to accept additional decoding work.
: <dfn attribute for=VideoDecoder>[[output callback]]</dfn>
:: Callback given at construction for decoded outputs.
: <dfn attribute for=VideoDecoder>[[error callback]]</dfn>
:: Callback given at construction for decode errors.
: <dfn attribute for=VideoDecoder>[[active decoder config]]</dfn>
:: The {{VideoDecoderConfig}} that is actively applied.
: <dfn attribute for=VideoDecoder>[[key chunk required]]</dfn>
:: A boolean indicating that the next chunk passed to {{VideoDecoder/decode()}}
    <em class="rfc2119">MUST</em> describe a [=key chunk=] as indicated by
    {{EncodedVideoChunk/type}}.
: <dfn attribute for=VideoDecoder>\[[state]]</dfn>
:: The current {{CodecState}} of this {{VideoDecoder}}.
: <dfn attribute for=VideoDecoder>\[[decodeQueueSize]]</dfn>
:: The number of pending decode requests. This number will decrease as the
    underlying codec is ready to accept new input.
: <dfn attribute for=VideoDecoder>[[pending flush promises]]</dfn>
:: A list of unresolved promises returned by calls to {{VideoDecoder/flush()}}.
: <dfn attribute for=VideoDecoder>[[dequeue event scheduled]]</dfn>
:: A boolean indicating whether a {{VideoDecoder/dequeue}} event is already
    scheduled to fire. Used to avoid event spam.


Constructors {#videodecoder-constructors}
-----------------------------------------
<dfn constructor for=VideoDecoder title="VideoDecoder(init)">
  VideoDecoder(init)
</dfn>
1. Let d be a new {{VideoDecoder}} object.
2. Assign a new [=queue=] to {{VideoDecoder/[[control message queue]]}}.
3. Assign `false` to {{VideoDecoder/[[message queue blocked]]}}.
4. Assign `null` to {{VideoDecoder/[[codec implementation]]}}.
5. Assign the result of starting a new [=parallel queue=] to
    {{VideoDecoder/[[codec work queue]]}}.
6. Assign `false` to {{VideoDecoder/[[codec saturated]]}}.
7. Assign init.output to {{VideoDecoder/[[output callback]]}}.
8. Assign init.error to {{VideoDecoder/[[error callback]]}}.
9. Assign `null` to {{VideoDecoder/[[active decoder config]]}}.
10. Assign `true` to {{VideoDecoder/[[key chunk required]]}}.
11. Assign `"unconfigured"` to {{VideoDecoder/[[state]]}}
12. Assign `0` to {{VideoDecoder/[[decodeQueueSize]]}}.
13. Assign a new [=list=] to {{VideoDecoder/[[pending flush promises]]}}.
14. Assign `false` to {{VideoDecoder/[[dequeue event scheduled]]}}.
15. Return d.

Attributes {#videodecoder-attributes}
-------------------------------------
: <dfn attribute for=VideoDecoder>state</dfn>
:: Returns the value of {{VideoDecoder/[[state]]}}.
: <dfn attribute for=VideoDecoder>decodeQueueSize</dfn>
:: Returns the value of {{VideoDecoder/[[decodeQueueSize]]}}.
: <dfn attribute for=VideoDecoder>ondequeue</dfn>
:: An [=event handler IDL attribute=] whose [=event handler event type=] is
    {{VideoDecoder/dequeue}}.

Event Summary {#videodecoder-event-summary}
-------------------------------------------
: <dfn event for=VideoDecoder>dequeue</dfn>
:: Fired at the {{VideoDecoder}} when the {{VideoDecoder/decodeQueueSize}} has
    decreased.

Methods {#videodecoder-methods}
-------------------------------
<dl>
  <dt><dfn method for=VideoDecoder>configure(config)</dfn></dt>
  <dd>
    [=Enqueues a control message=] to configure the video decoder for decoding
    chunks as described by |config|.

    NOTE: This method will trigger a {{NotSupportedError}} if the User Agent
      does not support |config|. Authors are encouraged to first check support
      by calling {{VideoDecoder/isConfigSupported()}} with |config|. User Agents
      don't have to support any particular codec type or configuration.

    When invoked, run these steps:
    1. If |config| is not a [=valid VideoDecoderConfig=], throw a
        {{TypeError}}.
    2. If {{VideoDecoder/[[state]]}} is `“closed”`, throw an
        {{InvalidStateError}}.
    3. Set {{VideoDecoder/[[state]]}} to `"configured"`.
    4. Set {{VideoDecoder/[[key chunk required]]}} to `true`.
    5. [=Queue a control message=] to configure the decoder with |config|.
    6. [=Process the control message queue=].

    [=Running a control message=] to configure the decoder means running
    these steps:
    1. Assign `true` to {{VideoDecoder/[[message queue blocked]]}}.
    2. Enqueue the following steps to {{VideoDecoder/[[codec work queue]]}}:
        1. Let |supported| be the result of running the <a>Check Configuration
            Support</a> algorithm with |config|.
        2. If |supported| is `false`, [=queue a task=] to run the <a>Close
            VideoDecoder</a> algorithm with {{NotSupportedError}} and abort
            these steps.
        3. If needed, assign {{VideoDecoder/[[codec implementation]]}} with an
            implementation supporting |config|.
        4. Configure {{VideoDecoder/[[codec implementation]]}} with |config|.
        5. [=queue a task=] to run the following steps:
            1. Assign `false` to {{VideoDecoder/[[message queue blocked]]}}.
            2. [=Queue a task=] to [=Process the control message queue=].
    3. Return `"processed"`.
  </dd>

  <dt><dfn method for=VideoDecoder>decode(chunk)</dfn></dt>
  <dd>
    [=Enqueues a control message=] to decode the given |chunk|.

    NOTE: Authors are encouraged to call {{VideoFrame/close()}} on output
        {{VideoFrame}}s immediately when frames are no longer needed. The
        underlying [=media resource=]s are owned by the {{VideoDecoder}} and
        failing to release them (or waiting for garbage collection) can cause
        decoding to stall.

    NOTE: {{VideoDecoder}} requires that frames are output in the order they
        expect to be presented, commonly known as presentation order. When using
        some {{VideoDecoder/[[codec implementation]]}}s the User Agent will have
        to reorder outputs into presentation order.

    When invoked, run these steps:
    1. If {{VideoDecoder/[[state]]}} is not `"configured"`, throw an
        {{InvalidStateError}}.
    2. If {{VideoDecoder/[[key chunk required]]}} is `true`:
        1. If |chunk|.{{EncodedVideoChunk/type}} is not
            {{EncodedVideoChunkType/key}}, throw a {{DataError}}.
        2. Implementers <em class="rfc2119">SHOULD</em> inspect the |chunk|'s
            {{EncodedVideoChunk/[[internal data]]}} to verify that
            it is truly a [=key chunk=]. If a mismatch is detected, throw a
            {{DataError}}.
        3. Otherwise, assign `false` to
            {{VideoDecoder/[[key chunk required]]}}.
    3. Increment {{VideoDecoder/[[decodeQueueSize]]}}.
    4. [=Queue a control message=] to decode the |chunk|.
    5. [=Process the control message queue=].

    [=Running a control message=] to decode the chunk means performing these steps:
    1. If {{VideoDecoder/[[codec saturated]]}} equals `true`, return `"not
        processed"`.
    2. If decoding chunk will cause the
        {{VideoDecoder/[[codec implementation]]}} to become [=saturated=],
        assign `true` to {{VideoDecoder/[[codec saturated]]}}.
    3. Decrement {{VideoDecoder/[[decodeQueueSize]]}} and run the
        [=VideoDecoder/Schedule Dequeue Event=] algorithm.
    4. Enqueue the following steps to the {{VideoDecoder/[[codec work queue]]}}:
        1. Attempt to use {{VideoDecoder/[[codec implementation]]}} to decode
            the chunk.
        2. If decoding results in an error, [=queue a task=] to run the
            [=Close VideoDecoder=] algorithm with {{EncodingError}} and return.
        3. If {{VideoDecoder/[[codec saturated]]}} equals `true` and
            {{VideoDecoder/[[codec implementation]]}} is no longer
            [=saturated=], [=queue a task=] to perform the following steps:
                1. Assign `false` to {{VideoDecoder/[[codec saturated]]}}.
                2. [=Process the control message queue=].
        4. Let |decoded outputs| be a [=list=] of decoded video data outputs
            emitted by {{VideoDecoder/[[codec implementation]]}} in
            presentation order.
        5. If |decoded outputs| is not empty, [=queue a task=] to run the
            [=Output VideoFrame=] algorithm with |decoded outputs|.
    5. Return `"processed"`.
  </dd>

  <dt><dfn method for=VideoDecoder>flush()</dfn></dt>
  <dd>
    Completes all [=control messages=] in the [=control message queue=]
    and emits all outputs.

    When invoked, run these steps:
    1. If {{VideoDecoder/[[state]]}} is not `"configured"`, return
        [=a promise rejected with=] {{InvalidStateError}} {{DOMException}}.
    2. Set {{VideoDecoder/[[key chunk required]]}} to `true`.
    3. Let |promise| be a new Promise.
    4. Append |promise| to {{VideoDecoder/[[pending flush promises]]}}.
    5. [=Queue a control message=] to flush the codec with |promise|.
    6. [=Process the control message queue=].
    7. Return |promise|.

    [=Running a control message=] to flush the codec means performing these
    steps with |promise|.
    1. Enqueue the following steps to the {{VideoDecoder/[[codec work queue]]}}:
        1. Signal {{VideoDecoder/[[codec implementation]]}} to emit all
            [=internal pending outputs=].
        2. Let |decoded outputs| be a [=list=] of decoded video data outputs
            emitted by {{VideoDecoder/[[codec implementation]]}}.
        3. [=Queue a task=] to perform these steps:
            1. If |decoded outputs| is not empty, run the [=Output VideoFrame=]
                algorithm with |decoded outputs|.
            2. Remove |promise| from
                {{VideoDecoder/[[pending flush promises]]}}.
            3. Resolve |promise|.
    2. Return `"processed"`.
  </dd>

  <dt><dfn method for=VideoDecoder>reset()</dfn></dt>
  <dd>
    Immediately resets all state including configuration,
    [=control messages=] in the [=control message queue=], and all pending
    callbacks.

    When invoked, run the [=Reset VideoDecoder=] algorithm with an
    {{AbortError}} {{DOMException}}.
  </dd>

  <dt><dfn method for=VideoDecoder>close()</dfn></dt>
  <dd>
    Immediately aborts all pending work and releases [=system resources=].
    Close is final.

    When invoked, run the [=Close VideoDecoder=] algorithm with an
    {{AbortError}} {{DOMException}}.
  </dd>

  <dt><dfn method for=VideoDecoder>isConfigSupported(config)</dfn></dt>
  <dd>
    Returns a promise indicating whether the provided |config| is supported by
    the User Agent.

    NOTE: The returned {{VideoDecoderSupport}} {{VideoDecoderSupport/config}}
        will contain only the dictionary members that User Agent recognized.
        Unrecognized dictionary members will be ignored. Authors can detect
        unrecognized dictionary members by comparing
        {{VideoDecoderSupport/config}} to their provided |config|.

    When invoked, run these steps:
    1. If |config| is not a <a>valid VideoDecoderConfig</a>, return
        [=a promise rejected with=] {{TypeError}}.
    2. Let |p| be a new Promise.
    3. Let |checkSupportQueue| be the result of starting a new <a>parallel
        queue</a>.
    4. Enqueue the following steps to |checkSupportQueue|:
        1. Let |supported| be the result of running
            the <a>Check Configuration Support</a> algorithm with |config|.
        2. [=Queue a task=] to run the following steps:
            1. Let |decoderSupport| be a newly constructed
                {{VideoDecoderSupport}}, initialized as follows:
                1. Set {{VideoDecoderSupport/config}} to the result of running the
                    <a>Clone Configuration</a> algorithm with |config|.
                2. Set {{VideoDecoderSupport/supported}} to |supported|.
            2. Resolve |p| with |decoderSupport|.
    5. Return |p|.
  </dd>
</dl>

Algorithms {#videodecoder-algorithms}
-------------------------------------
<dl>
  <dt><dfn for=VideoDecoder>Schedule Dequeue Event</dfn>
  <dd>
    1. If {{VideoDecoder/[[dequeue event scheduled]]}} equals `true`, return.
    2. Assign `true` to {{VideoDecoder/[[dequeue event scheduled]]}}.
    3. [=Queue a task=] to run the following steps:
        1. Fire a simple event named {{VideoDecoder/dequeue}} at [=this=].
        2. Assign `false` to {{VideoDecoder/[[dequeue event scheduled]]}}.
  </dd>
  <dt><dfn>Output VideoFrames</dfn> (with |outputs|)</dt>
  <dd>
    Run these steps:
    1. For each |output| in |outputs|:
        1. Let |timestamp| and |duration| be the
            {{EncodedVideoChunk/timestamp}} and {{EncodedVideoChunk/duration}}
            from the {{EncodedVideoChunk}} associated with |output|.
        2. Let |displayAspectWidth| and |displayAspectHeight| be undefined.
        3. If {{VideoDecoderConfig/displayAspectWidth}} and
            {{VideoDecoderConfig/displayAspectHeight}} [=map/exist=] in the
            {{VideoDecoder/[[active decoder config]]}}, assign their values to
            |displayAspectWidth| and |displayAspectHeight| respectively.
        4. Let |colorSpace| be the {{VideoColorSpace}} for |output| as detected
            by the codec implementation. If no {{VideoColorSpace}} is detected,
            let |colorSpace| be `undefined`.

            NOTE: The codec implementation can detect a {{VideoColorSpace}} by
                analyzing the bitstream. Detection is made on a best-effort
                basis. The exact method of detection is implementer defined and
                codec-specific.  Authors can override the detected
                {{VideoColorSpace}} by providing a
                {{VideoDecoderConfig/colorSpace}} in the {{VideoDecoderConfig}}.

        5. If {{VideoDecoderConfig/colorSpace}} [=map/exists=] in the
            {{VideoDecoder/[[active decoder config]]}}, assign its value to
            |colorSpace|.
        6. Assign the values of {{VideoDecoderConfig/rotation}} and
            {{VideoDecoderConfig/flip}} to |rotation| and |flip| respectively.
        7. Let |frame| be the result of running the [=Create a VideoFrame=]
            algorithm with |output|, |timestamp|, |duration|,
            |displayAspectWidth|, |displayAspectHeight|, |colorSpace|,
            |rotation|, and |flip|.
        8. Invoke {{VideoDecoder/[[output callback]]}} with |frame|.
  </dd>
  <dt><dfn>Reset VideoDecoder</dfn> (with |exception|)</dt>
  <dd>
    Run these steps:
    1. If {{VideoDecoder/state}} is `"closed"`, throw an {{InvalidStateError}}.
    2. Set {{VideoDecoder/state}} to `"unconfigured"`.
    3. Signal {{VideoDecoder/[[codec implementation]]}} to cease producing
        output for the previous configuration.
    4. Remove all [=control messages=] from the
        {{VideoDecoder/[[control message queue]]}}.
    5. If {{VideoDecoder/[[decodeQueueSize]]}} is greater than zero:
        1. Set {{VideoDecoder/[[decodeQueueSize]]}} to zero.
        2. Run the [=VideoDecoder/Schedule Dequeue Event=] algorithm.
    6. For each |promise| in {{VideoDecoder/[[pending flush promises]]}}:
        1. Reject |promise| with |exception|.
        2. Remove |promise| from {{VideoDecoder/[[pending flush promises]]}}.
  </dd>
  <dt><dfn>Close VideoDecoder</dfn> (with |exception|)</dt>
  <dd>
    Run these steps:
    1. Run the [=Reset VideoDecoder=] algorithm with |exception|.
    2. Set {{VideoDecoder/state}} to `"closed"`.
    3. Clear {{VideoDecoder/[[codec implementation]]}} and release associated
        [=system resources=].
    4. If |exception| is not an {{AbortError}} {{DOMException}},
        invoke the {{VideoDecoder/[[error callback]]}} with |exception|.
  </dd>
</dl>


AudioEncoder Interface {#audioencoder-interface}
================================================

<xmp class='idl'>
[Exposed=(Window,DedicatedWorker), SecureContext]
interface AudioEncoder : EventTarget {
  constructor(AudioEncoderInit init);

  readonly attribute CodecState state;
  readonly attribute unsigned long encodeQueueSize;
  attribute EventHandler ondequeue;

  undefined configure(AudioEncoderConfig config);
  undefined encode(AudioData data);
  Promise<undefined> flush();
  undefined reset();
  undefined close();

  static Promise<AudioEncoderSupport> isConfigSupported(AudioEncoderConfig config);
};

dictionary AudioEncoderInit {
  required EncodedAudioChunkOutputCallback output;
  required WebCodecsErrorCallback error;
};

callback EncodedAudioChunkOutputCallback =
    undefined (EncodedAudioChunk output,
               optional EncodedAudioChunkMetadata metadata = {});
</xmp>

Internal Slots {#audioencoder-internal-slots}
---------------------------------------------
: <dfn attribute for=AudioEncoder>[[control message queue]]</dfn>
:: A [=queue=] of [=control messages=] to be performed upon this [=codec=]
    instance. See [=[[control message queue]]=].
: <dfn attribute for=AudioEncoder>[[message queue blocked]]</dfn>
:: A boolean indicating when processing the
    {{AudioEncoder/[[control message queue]]}} is blocked by a pending
    [=control message=]. See [=[[message queue blocked]]=].
: <dfn attribute for=AudioEncoder>[[codec implementation]]</dfn>
:: Underlying encoder implementation provided by the User Agent. See
    [=[[codec implementation]]=].
: <dfn attribute for=AudioEncoder>[[codec work queue]]</dfn>
:: A [=parallel queue=] used for running parallel steps that reference the
    {{AudioEncoder/[[codec implementation]]}}. See [=[[codec work queue]]=].
: <dfn attribute for=AudioEncoder>[[codec saturated]]</dfn>
:: A boolean indicating when the {{AudioEncoder/[[codec implementation]]}} is
    unable to accept additional encoding work.
: <dfn attribute for=AudioEncoder>[[output callback]]</dfn>
:: Callback given at construction for encoded outputs.
: <dfn attribute for=AudioEncoder>[[error callback]]</dfn>
:: Callback given at construction for encode errors.
: <dfn attribute for=AudioEncoder>[[active encoder config]]</dfn>
:: The {{AudioEncoderConfig}} that is actively applied.
: <dfn attribute for=AudioEncoder>[[active output config]]</dfn>
:: The {{AudioDecoderConfig}} that describes how to decode the most recently
    emitted {{EncodedAudioChunk}}.
: <dfn attribute for=AudioEncoder>\[[state]]</dfn>
:: The current {{CodecState}} of this {{AudioEncoder}}.
: <dfn attribute for=AudioEncoder>\[[encodeQueueSize]]</dfn>
:: The number of pending encode requests. This number will decrease as the
    underlying codec is ready to accept new input.
: <dfn attribute for=AudioEncoder>[[pending flush promises]]</dfn>
:: A list of unresolved promises returned by calls to {{AudioEncoder/flush()}}.
: <dfn attribute for=AudioEncoder>[[dequeue event scheduled]]</dfn>
:: A boolean indicating whether a {{AudioEncoder/dequeue}} event is already
    scheduled to fire. Used to avoid event spam.

Constructors {#audioencoder-constructors}
-----------------------------------------
<dfn constructor for=AudioEncoder title="AudioEncoder(init)">
  AudioEncoder(init)
</dfn>
1. Let e be a new {{AudioEncoder}} object.
2. Assign a new [=queue=] to {{AudioEncoder/[[control message queue]]}}.
3. Assign `false` to {{AudioEncoder/[[message queue blocked]]}}.
4. Assign `null` to {{AudioEncoder/[[codec implementation]]}}.
5. Assign the result of starting a new [=parallel queue=] to
    {{AudioEncoder/[[codec work queue]]}}.
6. Assign `false` to {{AudioEncoder/[[codec saturated]]}}.
7. Assign init.output to {{AudioEncoder/[[output callback]]}}.
8. Assign init.error to {{AudioEncoder/[[error callback]]}}.
9. Assign `null` to {{AudioEncoder/[[active encoder config]]}}.
10. Assign `null` to {{AudioEncoder/[[active output config]]}}.
11. Assign `"unconfigured"` to {{AudioEncoder/[[state]]}}
12. Assign `0` to {{AudioEncoder/[[encodeQueueSize]]}}.
13. Assign a new [=list=] to {{AudioEncoder/[[pending flush promises]]}}.
14. Assign `false` to {{AudioEncoder/[[dequeue event scheduled]]}}.
15. Return e.

Attributes {#audioencoder-attributes}
-------------------------------------
: <dfn attribute for=AudioEncoder>state</dfn>
:: Returns the value of {{AudioEncoder/[[state]]}}.
: <dfn attribute for=AudioEncoder>encodeQueueSize</dfn>
:: Returns the value of {{AudioEncoder/[[encodeQueueSize]]}}.
: <dfn attribute for=AudioEncoder>ondequeue</dfn>
:: An [=event handler IDL attribute=] whose [=event handler event type=] is
    {{AudioEncoder/dequeue}}.

Event Summary {#audioencoder-event-summary}
-------------------------------------------
: <dfn event for=AudioEncoder>dequeue</dfn>
:: Fired at the {{AudioEncoder}} when the {{AudioEncoder/encodeQueueSize}} has
    decreased.

Methods {#audioencoder-methods}
-------------------------------
<dl>
  <dt><dfn method for=AudioEncoder>configure(config)</dfn></dt>
  <dd>
    [=Enqueues a control message=] to configure the audio encoder for
    encoding audio data as described by |config|.

    NOTE: This method will trigger a {{NotSupportedError}} if the User Agent
        does not support |config|. Authors are encouraged to first check support
        by calling {{AudioEncoder/isConfigSupported()}} with |config|. User
        Agents don't have to support any particular codec type or configuration.

    When invoked, run these steps:
    1. If |config| is not a [=valid AudioEncoderConfig=], throw a
        {{TypeError}}.
    2. If {{AudioEncoder/[[state]]}} is `"closed"`, throw an
        {{InvalidStateError}}.
    3. Set {{AudioEncoder/[[state]]}} to `"configured"`.
    4. [=Queue a control message=] to configure the encoder using |config|.
    5. [=Process the control message queue=].

    [=Running a control message=] to configure the encoder means performing
    these steps:
    1. Assign `true` to {{AudioEncoder/[[message queue blocked]]}}.
    2. Enqueue the following steps to {{AudioEncoder/[[codec work queue]]}}:
        1. Let |supported| be the result of running the <a>Check Configuration
            Support</a> algorithm with |config|.
        2. If |supported| is `false`, [=queue a task=] to run the <a>Close
            AudioEncoder</a> algorithm with {{NotSupportedError}} and abort
            these steps.
        3. If needed, assign {{AudioEncoder/[[codec implementation]]}} with an
            implementation supporting |config|.
        4. Configure {{AudioEncoder/[[codec implementation]]}} with |config|.
        5. [=queue a task=] to run the following steps:
            1. Assign `false` to {{AudioEncoder/[[message queue blocked]]}}.
            2. [=Queue a task=] to [=Process the control message queue=].
    3. Return `"processed"`.
  </dd>

  <dt><dfn method for=AudioEncoder>encode(data)</dfn></dt>
  <dd>
    [=Enqueues a control message=] to encode the given |data|.

    When invoked, run these steps:
    1. If the value of |data|'s {{platform object/[[Detached]]}} internal slot
        is `true`, throw a {{TypeError}}.
    2. If {{AudioEncoder/[[state]]}} is not `"configured"`, throw an
        {{InvalidStateError}}.
    3. Let |dataClone| hold the result of running the [=Clone AudioData=]
        algorithm with |data|.
    4. Increment {{AudioEncoder/[[encodeQueueSize]]}}.
    5. [=Queue a control message=] to encode |dataClone|.
    6. [=Process the control message queue=].

    [=Running a control message=] to encode the data means performing these
    steps:
    1. If {{AudioEncoder/[[codec saturated]]}} equals `true`, return `"not
        processed"`.
    2. If encoding |data| will cause the
        {{AudioEncoder/[[codec implementation]]}} to become [=saturated=],
        assign `true` to {{AudioEncoder/[[codec saturated]]}}.
    3. Decrement {{AudioEncoder/[[encodeQueueSize]]}} and run the
        [=AudioEncoder/Schedule Dequeue Event=] algorithm.
    4. Enqueue the following steps to the {{AudioEncoder/[[codec work queue]]}}:
        1. Attempt to use {{AudioEncoder/[[codec implementation]]}} to encode
            the [=media resource=] described by |dataClone|.
        2. If encoding results in an error, [=queue a task=] to run the
            [=Close AudioEncoder=] algorithm with {{EncodingError}} and return.
        3. If {{AudioEncoder/[[codec saturated]]}} equals `true` and
            {{AudioEncoder/[[codec implementation]]}} is no longer
            [=saturated=], [=queue a task=] to perform the following steps:
                1. Assign `false` to {{AudioEncoder/[[codec saturated]]}}.
                2. [=Process the control message queue=].
        4. Let |encoded outputs| be a [=list=] of encoded audio data outputs
            emitted by {{AudioEncoder/[[codec implementation]]}}.
        5. If |encoded outputs| is not empty, [=queue a task=] to run the
            [=Output EncodedAudioChunks=] algorithm with |encoded outputs|.
    5. Return `"processed"`.
  </dd>

  <dt><dfn method for=AudioEncoder>flush()</dfn></dt>
  <dd>
    Completes all [=control messages=] in the [=control message queue=]
    and emits all outputs.

    When invoked, run these steps:
    1. If {{AudioEncoder/[[state]]}} is not `"configured"`, return
        [=a promise rejected with=] {{InvalidStateError}} {{DOMException}}.
    2. Let |promise| be a new Promise.
    3. Append |promise| to {{AudioEncoder/[[pending flush promises]]}}.
    4. [=Queue a control message=] to flush the codec with |promise|.
    5. [=Process the control message queue=].
    6. Return |promise|.

    [=Running a control message=] to flush the codec means performing these
    steps with |promise|.
    1. Enqueue the following steps to the {{AudioEncoder/[[codec work queue]]}}:
        1. Signal {{AudioEncoder/[[codec implementation]]}} to emit all
            [=internal pending outputs=].
        2. Let |encoded outputs| be a [=list=] of encoded audio data outputs
            emitted by {{AudioEncoder/[[codec implementation]]}}.
        3. [=Queue a task=] to perform these steps:
            1. If |encoded outputs| is not empty, run the
                [=Output EncodedAudioChunks=] algorithm with |encoded outputs|.
            2. Remove |promise| from
                {{AudioEncoder/[[pending flush promises]]}}.
            3. Resolve |promise|.
    2. Return `"processed"`.
  </dd>

  <dt><dfn method for=AudioEncoder>reset()</dfn></dt>
  <dd>
    Immediately resets all state including configuration,
    [=control messages=] in the [=control message queue=], and all pending
    callbacks.

    When invoked, run the [=Reset AudioEncoder=] algorithm with an
    {{AbortError}} {{DOMException}}.
  </dd>

  <dt><dfn method for=AudioEncoder>close()</dfn></dt>
  <dd>
    Immediately aborts all pending work and releases [=system resources=].
    Close is final.

    When invoked, run the [=Close AudioEncoder=] algorithm with an
    {{AbortError}} {{DOMException}}.
  </dd>

  <dt><dfn method for=AudioEncoder>isConfigSupported(config)</dfn></dt>
  <dd>
    Returns a promise indicating whether the provided |config| is supported by
    the User Agent.

    NOTE: The returned {{AudioEncoderSupport}} {{AudioEncoderSupport/config}}
        will contain only the dictionary members that User Agent recognized.
        Unrecognized dictionary members will be ignored. Authors can detect
        unrecognized dictionary members by comparing
        {{AudioEncoderSupport/config}} to their provided |config|.

    When invoked, run these steps:
    1. If |config| is not a <a>valid AudioEncoderConfig</a>, return
        [=a promise rejected with=] {{TypeError}}.
    2. Let |p| be a new Promise.
    3. Let |checkSupportQueue| be the result of starting a new <a>parallel
        queue</a>.
    4. Enqueue the following steps to |checkSupportQueue|:
        1. Let |supported| be the result of running
            the <a>Check Configuration Support</a> algorithm with |config|.
        2. [=Queue a task=] to run the following steps:
            1. Let |encoderSupport| be a newly constructed
                {{AudioEncoderSupport}}, initialized as follows:
                1. Set {{AudioEncoderSupport/config}} to the result of running the
                    <a>Clone Configuration</a> algorithm with |config|.
                2. Set {{AudioEncoderSupport/supported}} to |supported|.
            2. Resolve |p| with |encoderSupport|.
    5. Return |p|.
  </dd>
</dl>

Algorithms {#audioencoder-algorithms}
-------------------------------------
<dl>
  <dt><dfn for=AudioEncoder>Schedule Dequeue Event</dfn>
  <dd>
    1. If {{AudioEncoder/[[dequeue event scheduled]]}} equals `true`, return.
    2. Assign `true` to {{AudioEncoder/[[dequeue event scheduled]]}}.
    3. [=Queue a task=] to run the following steps:
        1. Fire a simple event named {{AudioEncoder/dequeue}} at [=this=].
        2. Assign `false` to {{AudioEncoder/[[dequeue event scheduled]]}}.
  </dd>
  <dt><dfn>Output EncodedAudioChunks</dfn> (with |outputs|)</dt>
  <dd>
    Run these steps:
    1. For each |output| in |outputs|:
        1. Let |chunkInit| be an {{EncodedAudioChunkInit}} with the following
            keys:
            1. Let {{EncodedAudioChunkInit/data}} contain the encoded audio data
                from |output|.
            2. Let {{EncodedAudioChunkInit/type}} be the
                {{EncodedAudioChunkType}} of |output|.
            3. Let {{EncodedAudioChunkInit/timestamp}} be the
                {{AudioData/timestamp}} from the AudioData associated with
                |output|.
            4. Let {{EncodedAudioChunkInit/duration}} be the
                {{AudioData/duration}} from the AudioData associated with
                |output|.
        2. Let |chunk| be a new {{EncodedAudioChunk}} constructed with
            |chunkInit|.
        3. Let |chunkMetadata| be a new {{EncodedAudioChunkMetadata}}.
        4. Let |encoderConfig| be the
            {{AudioEncoder/[[active encoder config]]}}.
        5. Let |outputConfig| be a new {{AudioDecoderConfig}} that describes
            |output|. Initialize |outputConfig| as follows:
            1. Assign |encoderConfig|.{{AudioEncoderConfig/codec}} to
                |outputConfig|.{{AudioDecoderConfig/codec}}.
            2. Assign |encoderConfig|.{{AudioEncoderConfig/sampleRate}} to
                |outputConfig|.{{AudioDecoderConfig/sampleRate}}.
            3. Assign to
                |encoderConfig|.{{AudioEncoderConfig/numberOfChannels}} to
                |outputConfig|.{{AudioDecoderConfig/numberOfChannels}}.
            4. Assign |outputConfig|.{{AudioDecoderConfig/description}} with a
                sequence of codec specific bytes as determined by the
                {{AudioEncoder/[[codec implementation]]}}. The User Agent <em
                class="rfc2119">MUST</em> ensure that the provided description
                could be used to correctly decode output.

                NOTE: The codec specific requirements for populating the
                    {{AudioDecoderConfig/description}} are described in the
                    [[WEBCODECS-CODEC-REGISTRY]].

        6. If |outputConfig| and {{AudioEncoder/[[active output config]]}} are
            not [=equal dictionaries=]:
            1. Assign |outputConfig| to
                |chunkMetadata|.{{EncodedAudioChunkMetadata/decoderConfig}}.
            2. Assign |outputConfig| to
                {{AudioEncoder/[[active output config]]}}.
        7. Invoke {{AudioEncoder/[[output callback]]}} with |chunk| and
            |chunkMetadata|.
  </dd>
  <dt><dfn>Reset AudioEncoder</dfn> (with |exception|)</dt>
  <dd>
    Run these steps:
    1. If {{AudioEncoder/[[state]]}} is `"closed"`, throw an {{InvalidStateError}}.
    2. Set {{AudioEncoder/[[state]]}} to `"unconfigured"`.
    3. Set {{AudioEncoder/[[active encoder config]]}} to `null`.
    4. Set {{AudioEncoder/[[active output config]]}} to `null`.
    5. Signal {{AudioEncoder/[[codec implementation]]}} to cease producing
        output for the previous configuration.
    6. Remove all [=control messages=] from the
        {{AudioEncoder/[[control message queue]]}}.
    7. If {{AudioEncoder/[[encodeQueueSize]]}} is greater than zero:
        1. Set {{AudioEncoder/[[encodeQueueSize]]}} to zero.
        2. Run the [=AudioEncoder/Schedule Dequeue Event=] algorithm.
    8. For each |promise| in {{AudioEncoder/[[pending flush promises]]}}:
        1. Reject |promise| with |exception|.
        2. Remove |promise| from {{AudioEncoder/[[pending flush promises]]}}.
  </dd>
  <dt><dfn>Close AudioEncoder</dfn> (with |exception|)</dt>
  <dd>
    Run these steps:
    1. Run the [=Reset AudioEncoder=] algorithm with |exception|.
    2. Set {{AudioEncoder/[[state]]}} to `"closed"`.
    3. Clear {{AudioEncoder/[[codec implementation]]}} and release associated
        [=system resources=].
    4. If |exception| is not an {{AbortError}} {{DOMException}},
        invoke the {{AudioEncoder/[[error callback]]}} with |exception|.
  </dd>
</dl>

EncodedAudioChunkMetadata {#encoded-audio-chunk-metadata}
---------------------------------------------------------
The following metadata dictionary is emitted by the
{{EncodedAudioChunkOutputCallback}} alongside an associated
{{EncodedAudioChunk}}.

<xmp class='idl'>
dictionary EncodedAudioChunkMetadata {
  AudioDecoderConfig decoderConfig;
};
</xmp>

: <dfn dict-member for=EncodedAudioChunkMetadata>decoderConfig</dfn>
:: A {{AudioDecoderConfig}} that authors <em class="rfc2119">MAY</em> use to
    decode the associated {{EncodedAudioChunk}}.


VideoEncoder Interface {#videoencoder-interface}
================================================

<xmp class='idl'>
[Exposed=(Window,DedicatedWorker), SecureContext]
interface VideoEncoder : EventTarget {
  constructor(VideoEncoderInit init);

  readonly attribute CodecState state;
  readonly attribute unsigned long encodeQueueSize;
  attribute EventHandler ondequeue;

  undefined configure(VideoEncoderConfig config);
  undefined encode(VideoFrame frame, optional VideoEncoderEncodeOptions options = {});
  Promise<undefined> flush();
  undefined reset();
  undefined close();

  static Promise<VideoEncoderSupport> isConfigSupported(VideoEncoderConfig config);
};

dictionary VideoEncoderInit {
  required EncodedVideoChunkOutputCallback output;
  required WebCodecsErrorCallback error;
};

callback EncodedVideoChunkOutputCallback =
    undefined (EncodedVideoChunk chunk,
               optional EncodedVideoChunkMetadata metadata = {});
</xmp>

Internal Slots {#videoencoder-internal-slots}
---------------------------------------------
: <dfn attribute for=VideoEncoder>[[control message queue]]</dfn>
:: A [=queue=] of [=control messages=] to be performed upon this [=codec=]
    instance. See [=[[control message queue]]=].
: <dfn attribute for=VideoEncoder>[[message queue blocked]]</dfn>
:: A boolean indicating when processing the
    {{VideoEncoder/[[control message queue]]}} is blocked by a pending
    [=control message=]. See [=[[message queue blocked]]=].
: <dfn attribute for=VideoEncoder>[[codec implementation]]</dfn>
:: Underlying encoder implementation provided by the User Agent. See
    [=[[codec implementation]]=].
: <dfn attribute for=VideoEncoder>[[codec work queue]]</dfn>
:: A [=parallel queue=] used for running parallel steps that reference the
    {{VideoEncoder/[[codec implementation]]}}. See [=[[codec work queue]]=].
: <dfn attribute for=VideoEncoder>[[codec saturated]]</dfn>
:: A boolean indicating when the {{VideoEncoder/[[codec implementation]]}} is
    unable to accept additional encoding work.
: <dfn attribute for=VideoEncoder>[[output callback]]</dfn>
:: Callback given at construction for encoded outputs.
: <dfn attribute for=VideoEncoder>[[error callback]]</dfn>
:: Callback given at construction for encode errors.
: <dfn attribute for=VideoEncoder>[[active encoder config]]</dfn>
:: The {{VideoEncoderConfig}} that is actively applied.
: <dfn attribute for=VideoEncoder>[[active output config]]</dfn>
:: The {{VideoDecoderConfig}} that describes how to decode the most recently
    emitted {{EncodedVideoChunk}}.
: <dfn attribute for=VideoEncoder>\[[state]]</dfn>
:: The current {{CodecState}} of this {{VideoEncoder}}.
: <dfn attribute for=VideoEncoder>\[[encodeQueueSize]]</dfn>
:: The number of pending encode requests. This number will decrease as the
    underlying codec is ready to accept new input.
: <dfn attribute for=VideoEncoder>[[pending flush promises]]</dfn>
:: A list of unresolved promises returned by calls to {{VideoEncoder/flush()}}.
: <dfn attribute for=VideoEncoder>[[dequeue event scheduled]]</dfn>
:: A boolean indicating whether a {{VideoEncoder/dequeue}} event is already
    scheduled to fire. Used to avoid event spam.
: <dfn attribute for=VideoEncoder>[[active orientation]]</dfn>
:: An integer and boolean pair indicating the {{VideoFrame/[[flip]]}} and
    {{VideoFrame/[[rotation]]}} of the first {{VideoFrame}} given to
    {{VideoEncoder/encode()}} after {{VideoEncoder/configure()}}.

Constructors {#videoencoder-constructors}
-----------------------------------------
<dfn constructor for=VideoEncoder title="VideoEncoder(init)">
  VideoEncoder(init)
</dfn>
1. Let e be a new {{VideoEncoder}} object.
2. Assign a new [=queue=] to {{VideoEncoder/[[control message queue]]}}.
3. Assign `false` to {{VideoEncoder/[[message queue blocked]]}}.
4. Assign `null` to {{VideoEncoder/[[codec implementation]]}}.
5. Assign the result of starting a new [=parallel queue=] to
    {{VideoEncoder/[[codec work queue]]}}.
6. Assign `false` to {{VideoEncoder/[[codec saturated]]}}.
7. Assign init.output to {{VideoEncoder/[[output callback]]}}.
8. Assign init.error to {{VideoEncoder/[[error callback]]}}.
9. Assign `null` to {{VideoEncoder/[[active encoder config]]}}.
10. Assign `null` to {{VideoEncoder/[[active output config]]}}.
11. Assign `"unconfigured"` to {{VideoEncoder/[[state]]}}
12. Assign `0` to {{VideoEncoder/[[encodeQueueSize]]}}.
13. Assign a new [=list=] to {{VideoEncoder/[[pending flush promises]]}}.
14. Assign `false` to {{VideoEncoder/[[dequeue event scheduled]]}}.
15. Return e.

Attributes {#videoencoder-attributes}
-------------------------------------
: <dfn attribute for=VideoEncoder>state</dfn>
:: Returns the value of {{VideoEncoder/[[state]]}}.
: <dfn attribute for=VideoEncoder>encodeQueueSize</dfn>
:: Returns the value of {{VideoEncoder/[[encodeQueueSize]]}}.
: <dfn attribute for=VideoEncoder>ondequeue</dfn>
:: An [=event handler IDL attribute=] whose [=event handler event type=] is
    {{VideoEncoder/dequeue}}.

Event Summary {#videoencoder-event-summary}
-------------------------------------------
: <dfn event for=VideoEncoder>dequeue</dfn>
:: Fired at the {{VideoEncoder}} when the {{VideoEncoder/encodeQueueSize}} has
    decreased.

Methods {#videoencoder-methods}
-------------------------------
<dl>
  <dt><dfn method for=VideoEncoder>configure(config)</dfn></dt>
  <dd>
    [=Enqueues a control message=] to configure the video encoder for
    encoding video frames as described by |config|.

    NOTE: This method will trigger a {{NotSupportedError}} if the User Agent
        does not support |config|. Authors are encouraged to first check support
        by calling {{VideoEncoder/isConfigSupported()}} with |config|. User
        Agents don't have to support any particular codec type or
        configuration.

    When invoked, run these steps:
    1. If |config| is not a [=valid VideoEncoderConfig=], throw a
        {{TypeError}}.
    2. If {{VideoEncoder/[[state]]}} is `"closed"`, throw an
        {{InvalidStateError}}.
    3. Set {{VideoEncoder/[[state]]}} to `"configured"`.
    4. Set {{VideoEncoder/[[active orientation]]}} to `null`.
    5. [=Queue a control message=] to configure the encoder using |config|.
    6. [=Process the control message queue=].

    [=Running a control message=] to configure the encoder means performing
    these steps:
    1. Assign `true` to {{VideoEncoder/[[message queue blocked]]}}.
    2. Enqueue the following steps to {{VideoEncoder/[[codec work queue]]}}:
        1. Let |supported| be the result of running the <a>Check Configuration
            Support</a> algorithm with |config|.
        2. If |supported| is `false`, [=queue a task=] to run the <a>Close
            VideoEncoder</a> algorithm with {{NotSupportedError}} and abort
            these steps.
        3. If needed, assign {{VideoEncoder/[[codec implementation]]}} with an
            implementation supporting |config|.
        4. Configure {{VideoEncoder/[[codec implementation]]}} with |config|.
        5. [=queue a task=] to run the following steps:
            1. Assign `false` to {{VideoEncoder/[[message queue blocked]]}}.
            2. [=Queue a task=] to [=Process the control message queue=].
    3. Return `"processed"`.
  </dd>

  <dt><dfn method for=VideoEncoder>encode(|frame|, |options|)</dfn></dt>
  <dd>
    [=Enqueues a control message=] to encode the given |frame|.

    When invoked, run these steps:
    1. If the value of |frame|'s {{platform object/[[Detached]]}} internal slot
        is `true`, throw a {{TypeError}}.
    2. If {{VideoEncoder/[[state]]}} is not `"configured"`, throw an
        {{InvalidStateError}}.
    3. If {{VideoEncoder/[[active orientation]]}} is not `null` and does not match
        |frame|'s {{VideoFrame/[[rotation]]}} and {{VideoFrame/[[flip]]}} throw a
        {{DataError}}.
    4. If {{VideoEncoder/[[active orientation]]}} is `null`, set it to |frame|'s
        {{VideoFrame/[[rotation]]}} and {{VideoFrame/[[flip]]}}.
    5. Let |frameClone| hold the result of running the [=Clone VideoFrame=]
        algorithm with |frame|.
    6. Increment {{VideoEncoder/[[encodeQueueSize]]}}.
    7. [=Queue a control message=] to encode |frameClone|.
    8. [=Process the control message queue=].

    [=Running a control message=] to encode the frame means performing these
    steps:
    1. If {{VideoEncoder/[[codec saturated]]}} equals `true`, return `"not
        processed"`.
    2. If encoding |frame| will cause the
        {{VideoEncoder/[[codec implementation]]}} to become [=saturated=],
        assign `true` to {{VideoEncoder/[[codec saturated]]}}.
    3. Decrement {{VideoEncoder/[[encodeQueueSize]]}} and run the
        [=VideoEncoder/Schedule Dequeue Event=] algorithm.
    4. Enqueue the following steps to the {{VideoEncoder/[[codec work queue]]}}:
        1. Attempt to use {{VideoEncoder/[[codec implementation]]}} to encode
            the |frameClone| according to |options|.
        2. If encoding results in an error, [=queue a task=] to run the
            [=Close VideoEncoder=] algorithm with {{EncodingError}} and return.
        3. If {{VideoEncoder/[[codec saturated]]}} equals `true` and
            {{VideoEncoder/[[codec implementation]]}} is no longer
            [=saturated=], [=queue a task=] to perform the following steps:
                1. Assign `false` to {{VideoEncoder/[[codec saturated]]}}.
                2. [=Process the control message queue=].
        4. Let |encoded outputs| be a [=list=] of encoded video data outputs
            emitted by {{VideoEncoder/[[codec implementation]]}}.
        5. If |encoded outputs| is not empty, [=queue a task=] to run the
            [=Output EncodedVideoChunks=] algorithm with |encoded outputs|.
    5. Return `"processed"`.
  </dd>

  <dt><dfn method for=VideoEncoder>flush()</dfn></dt>
  <dd>
    Completes all [=control messages=] in the [=control message queue=]
    and emits all outputs.

    When invoked, run these steps:
    1. If {{VideoEncoder/[[state]]}} is not `"configured"`, return
        [=a promise rejected with=] {{InvalidStateError}} {{DOMException}}.
    2. Let |promise| be a new Promise.
    3. Append |promise| to {{VideoEncoder/[[pending flush promises]]}}.
    4. [=Queue a control message=] to flush the codec with |promise|.
    5. [=Process the control message queue=].
    6. Return |promise|.

    [=Running a control message=] to flush the codec means performing these
    steps with |promise|:
    1. Enqueue the following steps to the {{VideoEncoder/[[codec work queue]]}}:
        1. Signal {{VideoEncoder/[[codec implementation]]}} to emit all
            [=internal pending outputs=].
        2. Let |encoded outputs| be a [=list=] of encoded video data outputs
            emitted by {{VideoEncoder/[[codec implementation]]}}.
        3. [=Queue a task=] to perform these steps:
            1. If |encoded outputs| is not empty, run the
                [=Output EncodedVideoChunks=] algorithm with |encoded outputs|.
            2. Remove |promise| from
                {{VideoEncoder/[[pending flush promises]]}}.
            3. Resolve |promise|.
    2. Return `"processed"`.
  </dd>

  <dt><dfn method for=VideoEncoder>reset()</dfn></dt>
  <dd>
    Immediately resets all state including configuration,
    [=control messages=] in the [=control message queue=], and all pending
    callbacks.

    When invoked, run the [=Reset VideoEncoder=] algorithm with an
    {{AbortError}} {{DOMException}}.
  </dd>

  <dt><dfn method for=VideoEncoder>close()</dfn></dt>
  <dd>
    Immediately aborts all pending work and releases [=system resources=].
    Close is final.

    When invoked, run the [=Close VideoEncoder=] algorithm with an
    {{AbortError}} {{DOMException}}.
  </dd>

  <dt><dfn method for=VideoEncoder>isConfigSupported(config)</dfn></dt>
  <dd>
    Returns a promise indicating whether the provided |config| is supported by
    the User Agent.

    NOTE: The returned {{VideoEncoderSupport}} {{VideoEncoderSupport/config}}
        will contain only the dictionary members that User Agent recognized.
        Unrecognized dictionary members will be ignored. Authors can detect
        unrecognized dictionary members by comparing
        {{VideoEncoderSupport/config}} to their provided |config|.

    When invoked, run these steps:
    1. If |config| is not a <a>valid VideoEncoderConfig</a>, return
        [=a promise rejected with=] {{TypeError}}.
    2. Let |p| be a new Promise.
    3. Let |checkSupportQueue| be the result of starting a new <a>parallel
        queue</a>.
    4. Enqueue the following steps to |checkSupportQueue|:
        1. Let |supported| be the result of running
            the <a>Check Configuration Support</a> algorithm with |config|.
        2. [=Queue a task=] to run the following steps:
            1. Let |encoderSupport| be a newly constructed
                {{VideoEncoderSupport}}, initialized as follows:
                1. Set {{VideoEncoderSupport/config}} to the result of running the
                    <a>Clone Configuration</a> algorithm with |config|.
                2. Set {{VideoEncoderSupport/supported}} to |supported|.
           2. Resolve |p| with |encoderSupport|.
    5. Return |p|.
  </dd>
</dl>

Algorithms {#videoencoder-algorithms}
-------------------------------------
<dl>
  <dt><dfn for=VideoEncoder>Schedule Dequeue Event</dfn>
  <dd>
    1. If {{VideoEncoder/[[dequeue event scheduled]]}} equals `true`, return.
    2. Assign `true` to {{VideoEncoder/[[dequeue event scheduled]]}}.
    3. [=Queue a task=] to run the following steps:
        1. Fire a simple event named {{VideoEncoder/dequeue}} at [=this=].
        2. Assign `false` to {{VideoEncoder/[[dequeue event scheduled]]}}.
  </dd>
  <dt><dfn>Output EncodedVideoChunks</dfn> (with |outputs|)</dt>
  <dd>
    Run these steps:
    1. For each |output| in |outputs|:
        1. Let |chunkInit| be an {{EncodedVideoChunkInit}} with the following
            keys:
            1. Let {{EncodedVideoChunkInit/data}} contain the encoded video data
                from |output|.
            2. Let {{EncodedVideoChunkInit/type}} be the
                {{EncodedVideoChunkType}} of |output|.
            3. Let {{EncodedVideoChunkInit/timestamp}} be the
                {{VideoFrame/[[timestamp]]}} from the {{VideoFrame}}
                associated with |output|.
            4. Let {{EncodedVideoChunkInit/duration}} be the
                {{VideoFrame/[[duration]]}} from the {{VideoFrame}} associated
                with |output|.
        2. Let |chunk| be a new {{EncodedVideoChunk}} constructed with
            |chunkInit|.
        3. Let |chunkMetadata| be a new {{EncodedVideoChunkMetadata}}.
        4. Let |encoderConfig| be the
            {{VideoEncoder/[[active encoder config]]}}.
        5. Let |outputConfig| be a {{VideoDecoderConfig}} that describes
            |output|. Initialize |outputConfig| as follows:
            1. Assign `encoderConfig.codec` to `outputConfig.codec`.
            2. Assign `encoderConfig.width` to
                `outputConfig.codedWidth`.
            3. Assign `encoderConfig.height` to
                `outputConfig.codedHeight`.
            4. Assign `encoderConfig.displayWidth` to
                `outputConfig.displayAspectWidth`.
            5. Assign `encoderConfig.displayHeight` to
                `outputConfig.displayAspectHeight`.
            6. Assign {{VideoFrame/[[rotation]]}} from the {{VideoFrame}}
                associated with |output| to `outputConfig.rotation`.
            7. Assign {{VideoFrame/[[flip]]}} from the {{VideoFrame}}
                associated with |output| to `outputConfig.flip`.
            8. Assign the remaining keys of `outputConfig` as determined by
                {{VideoEncoder/[[codec implementation]]}}. The User Agent
                <em class="rfc2119">MUST</em> ensure that the configuration is
                completely described such that |outputConfig| could be used to
                correctly decode |output|.

                NOTE: The codec specific requirements for populating the
                    {{VideoDecoderConfig/description}} are described in the
                    [[WEBCODECS-CODEC-REGISTRY]].

        6. If |outputConfig| and {{VideoEncoder/[[active output config]]}} are
            not <a>equal dictionaries</a>:
            1. Assign |outputConfig| to
                |chunkMetadata|.{{EncodedVideoChunkMetadata/decoderConfig}}.
            2. Assign |outputConfig| to
                {{VideoEncoder/[[active output config]]}}.
        7. If |encoderConfig|.{{VideoEncoderConfig/scalabilityMode}}
            describes multiple [=temporal layers=]:
            1. Let |svc| be a new {{SvcOutputMetadata}} instance.
            2. Let |temporal_layer_id| be the zero-based index describing the
                temporal layer for |output|.
            3. Assign |temporal_layer_id| to
                |svc|.{{SvcOutputMetadata/temporalLayerId}}.
            4. Assign |svc| to
                |chunkMetadata|.{{EncodedVideoChunkMetadata/svc}}.
        8. If |encoderConfig|.{{VideoEncoderConfig/alpha}} is set to `"keep"`:
            1. Let |alphaSideData| be the encoded alpha data in |output|.
            2. Assign |alphaSideData| to
                |chunkMetadata|.{{EncodedVideoChunkMetadata/alphaSideData}}.
        9. Invoke {{VideoEncoder/[[output callback]]}} with |chunk| and
            |chunkMetadata|.
  </dd>
  <dt><dfn>Reset VideoEncoder</dfn> (with |exception|)</dt>
  <dd>
    Run these steps:
    1. If {{VideoEncoder/[[state]]}} is `"closed"`, throw an {{InvalidStateError}}.
    2. Set {{VideoEncoder/[[state]]}} to `"unconfigured"`.
    3. Set {{VideoEncoder/[[active encoder config]]}} to `null`.
    4. Set {{VideoEncoder/[[active output config]]}} to `null`.
    5. Signal {{VideoEncoder/[[codec implementation]]}} to cease producing
        output for the previous configuration.
    6. Remove all [=control messages=] from the
        {{VideoEncoder/[[control message queue]]}}.
    7. If {{VideoEncoder/[[encodeQueueSize]]}} is greater than zero:
        1. Set {{VideoEncoder/[[encodeQueueSize]]}} to zero.
        2. Run the [=VideoEncoder/Schedule Dequeue Event=] algorithm.
    8. For each |promise| in {{VideoEncoder/[[pending flush promises]]}}:
        1. Reject |promise| with |exception|.
        2. Remove |promise| from {{VideoEncoder/[[pending flush promises]]}}.
  </dd>
  <dt><dfn>Close VideoEncoder</dfn> (with |exception|)</dt>
  <dd>
    Run these steps:
    1. Run the [=Reset VideoEncoder=] algorithm with |exception|.
    2. Set {{VideoEncoder/[[state]]}} to `"closed"`.
    3. Clear {{VideoEncoder/[[codec implementation]]}} and release associated
        [=system resources=].
    4. If |exception| is not an {{AbortError}} {{DOMException}},
        invoke the {{VideoEncoder/[[error callback]]}} with |exception|.
  </dd>
</dl>

EncodedVideoChunkMetadata {#encoded-video-chunk-metadata}
---------------------------------------------------------
The following metadata dictionary is emitted by the
{{EncodedVideoChunkOutputCallback}} alongside an associated
{{EncodedVideoChunk}}.

<xmp class='idl'>
dictionary EncodedVideoChunkMetadata {
  VideoDecoderConfig decoderConfig;
  SvcOutputMetadata svc;
  BufferSource alphaSideData;
};

dictionary SvcOutputMetadata {
  unsigned long temporalLayerId;
};
</xmp>

: <dfn dict-member for=EncodedVideoChunkMetadata>decoderConfig</dfn>
:: A {{VideoDecoderConfig}} that authors <em class="rfc2119">MAY</em> use to
    decode the associated {{EncodedVideoChunk}}.

: <dfn dict-member for=EncodedVideoChunkMetadata>svc</dfn>
:: A collection of metadata describing this {{EncodedVideoChunk}} with respect
    to the configured {{VideoEncoderConfig/scalabilityMode}}.

: <dfn dict-member for=EncodedVideoChunkMetadata>alphaSideData</dfn>
:: A {{BufferSource}} that contains the {{EncodedVideoChunk}}'s extra alpha
    channel data.

: <dfn dict-member for=SvcOutputMetadata>temporalLayerId</dfn>
:: A number that identifies the [=temporal layer=] for the associated
    {{EncodedVideoChunk}}.


Configurations{#configurations}
===============================

<dfn>Check Configuration Support</dfn> (with |config|) {#config-support}
------------------------------------------------------------------------
Run these steps:
1. If the <a>codec string</a> in |config|.codec is not a
    <a>valid codec string</a> or is otherwise unrecognized by the User Agent,
    return `false`.
2. If |config| is an {{AudioDecoderConfig}} or {{VideoDecoderConfig}} and the
    User Agent can't provide a <a>codec</a> that can decode the exact profile
    (where present), level (where present), and constraint bits (where present)
    indicated by the <a>codec string</a> in |config|.codec, return `false`.
3. If |config| is an {{AudioEncoderConfig}} or {{VideoEncoderConfig}}:
    1. If the <a>codec string</a> in |config|.codec contains a profile and the
        User Agent can't provide a <a>codec</a> that can encode the exact
        profile indicated by |config|.codec, return `false`.
    2. If the <a>codec string</a> in |config|.codec contains a level and the
        User Agent can't provide a <a>codec</a> that can encode to a level less
        than or equal to the level indicated by |config|.codec, return `false`.
    3. If the <a>codec string</a> in |config|.codec contains constraint bits and
        the User Agent can't provide a <a>codec</a> that can produce an encoded
        bitstream at least as constrained as indicated by |config|.codec, return
        `false`.
4. If the User Agent can provide a <a>codec</a> to support all entries of the
    |config|, including applicable default values for keys that are not
    included, return `true`.

    NOTE: The types {{AudioDecoderConfig}}, {{VideoDecoderConfig}},
        {{AudioEncoderConfig}}, and {{VideoEncoderConfig}} each define their
        respective configuration entries and defaults.

    NOTE: Support for a given configuration can change dynamically if the
        hardware is altered (e.g. external GPU unplugged) or if essential
        hardware resources are exhausted. User Agents describe support on
        a best-effort basis given the resources that are available at the time
        of the query.

5. Otherwise, return false.

<dfn>Clone Configuration</dfn> (with |config|) {#clone-config}
--------------------------------------------------------------

NOTE: This algorithm will copy only the dictionary members that the User Agent
    recognizes as part of the dictionary type.

Run these steps:
1. Let |dictType| be the type of dictionary |config|.
2. Let <var ignore=''>clone</var> be a new empty instance of |dictType|.
3. For each dictionary member |m| defined on |dictType|:
    1. If |m| does not [=map/exist=] in |config|, then [=iteration/continue=].
    2. If `config[m]` is a nested dictionary, set `clone[m]` to the result of
        recursively running the <a>Clone Configuration</a> algorithm with
        `config[m]`.
    3. Otherwise, assign a copy of `config[m]` to `clone[m]`.

Note: This implements a "deep-copy". These configuration objects are
frequently used as the input of asynchronous operations. Copying means that
modifying the original object while the operation is in flight won't change the
operation's outcome.


Signalling Configuration Support{#config-support-info}
------------------------------------------------------

### AudioDecoderSupport ### {#audio-decoder-support}
<xmp class='idl'>
dictionary AudioDecoderSupport {
  boolean supported;
  AudioDecoderConfig config;
};
</xmp>

<dl>
  <dt><dfn dict-member for=AudioDecoderSupport>supported</dfn></dt>
  <dd>
    A boolean indicating the whether the corresponding
    {{AudioDecoderSupport/config}} is supported by the User Agent.
  </dd>
  <dt><dfn dict-member for=AudioDecoderSupport>config</dfn></dt>
  <dd>
    An {{AudioDecoderConfig}} used by the User Agent in determining the value of
    {{AudioDecoderSupport/supported}}.
  </dd>
</dl>

### VideoDecoderSupport ### {#video-decoder-support}
<xmp class='idl'>
dictionary VideoDecoderSupport {
  boolean supported;
  VideoDecoderConfig config;
};
</xmp>

<dl>
  <dt><dfn dict-member for=VideoDecoderSupport>supported</dfn></dt>
  <dd>
    A boolean indicating the whether the corresponding
    {{VideoDecoderSupport/config}} is supported by the User Agent.
  </dd>
  <dt><dfn dict-member for=VideoDecoderSupport>config</dfn></dt>
  <dd>
    A {{VideoDecoderConfig}} used by the User Agent in determining the value of
    {{VideoDecoderSupport/supported}}.
  </dd>
</dl>

### AudioEncoderSupport ### {#audio-encoder-support}
<xmp class='idl'>
dictionary AudioEncoderSupport {
  boolean supported;
  AudioEncoderConfig config;
};
</xmp>

<dl>
  <dt><dfn dict-member for=AudioEncoderSupport>supported</dfn></dt>
  <dd>
    A boolean indicating the whether the corresponding
    {{AudioEncoderSupport/config}} is supported by the User Agent.
  </dd>
  <dt><dfn dict-member for=AudioEncoderSupport>config</dfn></dt>
  <dd>
    An {{AudioEncoderConfig}} used by the User Agent in determining the value of
    {{AudioEncoderSupport/supported}}.
  </dd>
</dl>

### VideoEncoderSupport ### {#video-encoder-support}
<xmp class='idl'>
dictionary VideoEncoderSupport {
  boolean supported;
  VideoEncoderConfig config;
};
</xmp>

<dl>
  <dt><dfn dict-member for=VideoEncoderSupport>supported</dfn></dt>
  <dd>
    A boolean indicating the whether the corresponding
    {{VideoEncoderSupport/config}} is supported by the User Agent.
  </dd>
  <dt><dfn dict-member for=VideoEncoderSupport>config</dfn></dt>
  <dd>
    A {{VideoEncoderConfig}} used by the User Agent in determining the value of
    {{VideoEncoderSupport/supported}}.
  </dd>
</dl>

<dfn export>Codec String</dfn>{#config-codec-string}
----------------------------------------------------
A codec string describes a given codec format to be used for encoding or
decoding.

A <dfn>valid codec string</dfn> <em class="rfc2119">MUST</em> meet the following
conditions.
1. Is valid per the relevant codec specification (see examples below).
2. It describes a single codec.
3. It is unambiguous about codec profile, level, and constraint bits for codecs
    that define these concepts.

NOTE: In other media specifications, codec strings historically accompanied a
    [=MIME type=] as the "codecs=" parameter
    ({{MediaSource/isTypeSupported()}}, {{HTMLMediaElement/canPlayType()}})
    [[RFC6381]]. In this specification, encoded media is not containerized;
    hence, only the value of the codecs parameter is accepted.

NOTE: Encoders for codecs that define level and constraint bits have flexibility
    around these parameters, but won't produce bitstreams that have a higher
    level or are less constrained than requested.

The format and semantics for codec strings are defined by codec registrations
listed in the [[WEBCODECS-CODEC-REGISTRY]]. A compliant implementation <em
class="rfc2119">MAY</em> support any combination of codec registrations or
none at all.

AudioDecoderConfig{#audio-decoder-config}
-----------------------------------------
<xmp class='idl'>
dictionary AudioDecoderConfig {
  required DOMString codec;
  [EnforceRange] required unsigned long sampleRate;
  [EnforceRange] required unsigned long numberOfChannels;
  AllowSharedBufferSource description;
};
</xmp>

To check if an {{AudioDecoderConfig}} is a <dfn export>valid AudioDecoderConfig</dfn>,
    run these steps:
1. If {{AudioDecoderConfig/codec}} is empty after
    [=strip leading and trailing ASCII whitespace|stripping leading and trailing ASCII whitespace=],
    return `false`.
2. If {{AudioDecoderConfig/description}} is [[=BufferSource/detached=]], return
     false.
3. Return `true`.

<dl>
  <dt><dfn dict-member for=AudioDecoderConfig>codec</dfn></dt>
  <dd>Contains a <a>codec string</a> in |config|.codec describing the codec.</dd>

  <dt><dfn dict-member for=AudioDecoderConfig>sampleRate</dfn></dt>
  <dd>The number of frame samples per second.</dd>

  <dt><dfn dict-member for=AudioDecoderConfig>numberOfChannels</dfn></dt>
  <dd>The number of audio channels.</dd>

  <dt><dfn dict-member for=AudioDecoderConfig>description</dfn></dt>
  <dd>
    A sequence of codec specific bytes, commonly known as extradata.

    NOTE: The registrations in the [[WEBCODECS-CODEC-REGISTRY]] describe whether/how to
        populate this sequence, corresponding to the provided
        {{AudioDecoderConfig/codec}}.
  </dd>
</dl>


VideoDecoderConfig{#video-decoder-config}
-----------------------------------------
<xmp class='idl'>
dictionary VideoDecoderConfig {
  required DOMString codec;
  AllowSharedBufferSource description;
  [EnforceRange] unsigned long codedWidth;
  [EnforceRange] unsigned long codedHeight;
  [EnforceRange] unsigned long displayAspectWidth;
  [EnforceRange] unsigned long displayAspectHeight;
  VideoColorSpaceInit colorSpace;
  HardwareAcceleration hardwareAcceleration = "no-preference";
  boolean optimizeForLatency;
  double rotation = 0;
  boolean flip = false;
};
</xmp>

To check if a {{VideoDecoderConfig}} is a <dfn export>valid VideoDecoderConfig</dfn>,
run these steps:
1. If {{VideoDecoderConfig/codec}} is empty after
    [=strip leading and trailing ASCII whitespace|stripping leading and trailing ASCII whitespace=],
    return `false`.
2. If one of {{VideoDecoderConfig/codedWidth}} or
    {{VideoDecoderConfig/codedHeight}} is provided but the other isn't,
    return `false`.
3. If {{VideoDecoderConfig/codedWidth}} = 0 or
    {{VideoDecoderConfig/codedHeight}} = 0, return `false`.
4. If one of {{VideoDecoderConfig/displayAspectWidth}} or
    {{VideoDecoderConfig/displayAspectHeight}} is provided but the other isn't,
    return `false`.
5. If {{VideoDecoderConfig/displayAspectWidth}} = 0 or
    {{VideoDecoderConfig/displayAspectHeight}} = 0, return `false`.
6. If {{VideoDecoderConfig/description}} is [[=BufferSource/detached=]],
    return false.
7. Return `true`.

<dl>
  <dt><dfn dict-member for=VideoDecoderConfig>codec</dfn></dt>
  <dd>Contains a <a>codec string</a> describing the codec.</dd>
  <dt><dfn dict-member for=VideoDecoderConfig>description</dfn></dt>
  <dd>
    A sequence of codec specific bytes, commonly known as extradata.

    NOTE: The registrations in the [[WEBCODECS-CODEC-REGISTRY]] describes
        whether/how to populate this sequence, corresponding to the provided
        {{VideoDecoderConfig/codec}}.
  </dd>

  <dt><dfn dict-member for=VideoDecoderConfig>codedWidth</dfn></dt>
  <dd>
    Width of the VideoFrame in pixels, potentially including non-visible
    padding, and prior to considering potential ratio adjustments.
  </dd>

  <dt><dfn dict-member for=VideoDecoderConfig>codedHeight</dfn></dt>
  <dd>
    Height of the VideoFrame in pixels, potentially including non-visible
    padding, and prior to considering potential ratio adjustments.

    NOTE: {{VideoDecoderConfig/codedWidth}} and {{VideoDecoderConfig/codedHeight}}
    are used when selecting a {{VideoDecoder/[[codec implementation]]}}.
  </dd>

  <dt><dfn dict-member for=VideoDecoderConfig>displayAspectWidth</dfn></dt>
  <dd>
    Horizontal dimension of the VideoFrame's aspect ratio when displayed.
  </dd>

  <dt><dfn dict-member for=VideoDecoderConfig>displayAspectHeight</dfn></dt>
  <dd>
    Vertical dimension of the VideoFrame's aspect ratio when displayed.

    NOTE: {{VideoFrame/displayWidth}} and {{VideoFrame/displayHeight}} can both be
      different from {{VideoDecoderConfig/displayAspectWidth}} and
      {{VideoDecoderConfig/displayAspectHeight}}, but have identical ratios, after scaling is applied when [=create a videoframe|creating the
      video frame=].
  </dd>

  <dt><dfn dict-member for=VideoDecoderConfig>colorSpace</dfn></dt>
  <dd>
    Configures the {{VideoFrame}}.{{VideoFrame/colorSpace}} for {{VideoFrame}}s
    associated with this {{VideoDecoderConfig}}. If
    {{VideoDecoderConfig/colorSpace}} [=map/exists=], the provided values will
    override any in-band values from the bitsream.
  </dd>

  <dt><dfn dict-member for=VideoDecoderConfig>hardwareAcceleration</dfn></dt>
  <dd>
    Hint that configures hardware acceleration for this codec. See
    {{HardwareAcceleration}}.
  </dd>

  <dt><dfn dict-member for=VideoDecoderConfig>optimizeForLatency</dfn></dt>
  <dd>
    Hint that the selected decoder <em class="rfc2119">SHOULD</em> be configured
    to minimize the number of {{EncodedVideoChunk}}s that have to be decoded
    before a {{VideoFrame}} is output.

    NOTE: In addition to User Agent and hardware limitations, some codec
        bitstreams require a minimum number of inputs before any output can be
        produced.
  </dd>
  <dt><dfn dict-member for=VideoDecoderConfig>rotation</dfn></dt>
  <dd>
    Sets the {{VideoFrame/rotation}} attribute on decoded frames.
  </dd>
  <dt><dfn dict-member for=VideoDecoderConfig>flip</dfn></dt>
  <dd>
    Sets the {{VideoFrame/flip}} attribute on decoded frames.
  </dd>
</dl>


AudioEncoderConfig{#audio-encoder-config}
-----------------------------------------
<xmp class='idl'>
dictionary AudioEncoderConfig {
  required DOMString codec;
  [EnforceRange] required unsigned long sampleRate;
  [EnforceRange] required unsigned long numberOfChannels;
  [EnforceRange] unsigned long long bitrate;
  BitrateMode bitrateMode = "variable";
};
</xmp>

NOTE: Codec-specific extensions to {{AudioEncoderConfig}} are described in
    their registrations in the [[WEBCODECS-CODEC-REGISTRY]].

To check if an {{AudioEncoderConfig}} is a <dfn>valid AudioEncoderConfig</dfn>,
run these steps:
1. If {{AudioEncoderConfig/codec}} is empty after
    [=strip leading and trailing ASCII whitespace|stripping leading and trailing ASCII whitespace=],
    return `false`.
2. If the {{AudioEncoderConfig}} has a codec-specific extension and the corresponding
    registration in the [[WEBCODECS-CODEC-REGISTRY]] defines steps to check whether
    the extension is a valid extension, return the result of running those steps.
3. If {{AudioEncoderConfig/sampleRate}} or {{AudioEncoderConfig/numberOfChannels}} are
     equal to zero, return `false`.
4. Return `true`.

<dl>
  <dt><dfn dict-member for=AudioEncoderConfig>codec</dfn></dt>
  <dd>Contains a <a>codec string</a> describing the codec.</dd>

  <dt><dfn dict-member for=AudioEncoderConfig>sampleRate</dfn></dt>
  <dd>The number of frame samples per second.</dd>

  <dt><dfn dict-member for=AudioEncoderConfig>numberOfChannels</dfn></dt>
  <dd>The number of audio channels.</dd>

  <dt><dfn dict-member for=AudioEncoderConfig>bitrate</dfn></dt>
  <dd>
    The average bitrate of the encoded audio given in units of bits per second.
  </dd>
  <dt><dfn dict-member for=AudioEncoderConfig>bitrateMode</dfn></dt>
  <dd>
    Configures the encoder to use a {{BitrateMode/constant}} or
    {{BitrateMode/variable}} bitrate as defined by [[MEDIASTREAM-RECORDING]].

    NOTE: Not all audio codecs support specific {{BitrateMode}}s, Authors are
    encouraged to check by calling {{AudioEncoder/isConfigSupported()}} with
    |config|.
  </dd>
</dl>


VideoEncoderConfig{#video-encoder-config}
-----------------------------------------
<xmp class='idl'>
dictionary VideoEncoderConfig {
  required DOMString codec;
  [EnforceRange] required unsigned long width;
  [EnforceRange] required unsigned long height;
  [EnforceRange] unsigned long displayWidth;
  [EnforceRange] unsigned long displayHeight;
  [EnforceRange] unsigned long long bitrate;
  double framerate;
  HardwareAcceleration hardwareAcceleration = "no-preference";
  AlphaOption alpha = "discard";
  DOMString scalabilityMode;
  VideoEncoderBitrateMode bitrateMode = "variable";
  LatencyMode latencyMode = "quality";
  DOMString contentHint;
};
</xmp>

NOTE: Codec-specific extensions to {{VideoEncoderConfig}} are described in their
    registrations in the [[WEBCODECS-CODEC-REGISTRY]].

To check if a {{VideoEncoderConfig}} is a <dfn>valid VideoEncoderConfig</dfn>,
    run these steps:
1. If {{VideoEncoderConfig/codec}} is empty after
    [=strip leading and trailing ASCII whitespace|stripping leading and trailing ASCII whitespace=],
    return `false`.
2. If {{VideoEncoderConfig/width}} = 0 or {{VideoEncoderConfig/height}}
    = 0, return `false`.
3. If {{VideoEncoderConfig/displayWidth}} = 0 or
    {{VideoEncoderConfig/displayHeight}} = 0, return `false`.
4. Return `true`.

<dl>
  <dt><dfn dict-member for=VideoEncoderConfig>codec</dfn></dt>
  <dd>Contains a <a>codec string</a> in |config|.codec describing the codec.</dd>

  <dt><dfn dict-member for=VideoEncoderConfig>width</dfn></dt>
  <dd>
    The encoded width of output {{EncodedVideoChunk}}s in pixels, prior to any
    display aspect ratio adjustments.

    The encoder <em class="rfc2119">MUST</em> scale any {{VideoFrame}} whose
    {{VideoFrame/[[visible width]]}} differs from this value.
  </dd>

  <dt><dfn dict-member for=VideoEncoderConfig>height</dfn></dt>
  <dd>
    The encoded height of output {{EncodedVideoChunk}}s in pixels, prior to any
    display aspect ratio adjustments.

    The encoder <em class="rfc2119">MUST</em> scale any {{VideoFrame}} whose
    {{VideoFrame/[[visible height]]}} differs from this value.
  </dd>
</dl>

<dl>
  <dt><dfn dict-member for=VideoEncoderConfig>displayWidth</dfn></dt>
  <dd>
    The intended display width of output {{EncodedVideoChunk}}s in pixels.
    Defaults to {{VideoEncoderConfig/width}} if not present.
  </dd>

  <dt><dfn dict-member for=VideoEncoderConfig>displayHeight</dfn></dt>
  <dd>
    The intended display height of output {{EncodedVideoChunk}}s in pixels.
    Defaults to {{VideoEncoderConfig/width}} if not present.
  </dd>
</dl>

<div class='note'>
  NOTE: Providing a {{VideoEncoderConfig/displayWidth}} or
      {{VideoEncoderConfig/displayHeight}} that differs from
      {{VideoEncoderConfig/width}} and {{VideoEncoderConfig/height}} signals
      that chunks are to be scaled after decoding to arrive at the final
      display aspect ratio.

      For many codecs this is merely pass-through information, but some codecs
      can sometimes include display sizing in the bitstream.
</div>

<dl>
  <dt><dfn dict-member for=VideoEncoderConfig>bitrate</dfn></dt>
  <dd>
    The average bitrate of the encoded video given in units of bits per second.

    NOTE: Authors are encouraged to additionally provide a
    {{VideoEncoderConfig/framerate}} to inform rate control.
  </dd>

  <dt><dfn dict-member for=VideoEncoderConfig>framerate</dfn></dt>
  <dd>
    The expected frame rate in frames per second, if known. This value, along
    with the frame {{VideoFrame/timestamp}}, <em class="rfc2119">SHOULD</em> be
    used by the video encoder to calculate the optimal byte length for each
    encoded frame. Additionally, the value <em class="rfc2119">SHOULD</em> be
    considered a target deadline for outputting encoding chunks when
    {{VideoEncoderConfig/latencyMode}} is set to {{LatencyMode/realtime}}.
  </dd>

  <dt><dfn dict-member for=VideoEncoderConfig>hardwareAcceleration</dfn></dt>
  <dd>
    Hint that configures hardware acceleration for this codec. See
    {{HardwareAcceleration}}.
  </dd>

<dt><dfn dict-member for=VideoEncoderConfig>alpha</dfn></dt>
  <dd>
    Whether the alpha component of the {{VideoFrame}} inputs <em
    class="rfc2119">SHOULD</em> be kept or discarded prior to encoding. If
    {{VideoEncoderConfig/alpha}} is equal to {{AlphaOption/discard}}, alpha data
    is always discarded, regardless of a {{VideoFrame}}'s
    {{VideoFrame/[[format]]}}.
  </dd>

  <dt><dfn dict-member for=VideoEncoderConfig>scalabilityMode</dfn></dt>
  <dd>
    An encoding [=scalability mode identifier=] as defined by [[WebRTC-SVC]].
  </dd>

  <dt><dfn dict-member for=VideoEncoderConfig>bitrateMode</dfn></dt>
  <dd>
    Configures encoding to use one of the rate control modes specified by
    {{VideoEncoderBitrateMode}}.

    NOTE: The precise degree of bitrate fluctuation in either mode is
        implementation defined.
  </dd>

  <dt><dfn dict-member for=VideoEncoderConfig>latencyMode</dfn></dt>
  <dd>
    Configures latency related behaviors for this codec. See {{LatencyMode}}.
  </dd>
  <dt><dfn dict-member for=VideoEncoderConfig>contentHint</dfn></dt>
  <dd>
    An encoding [=video content hint=] as defined by [[mst-content-hint]].

    The User Agent <em class="rfc2119">MAY</em> use this hint to set
    expectations about incoming {{VideoFrame}}s and to improve encoding quality.
    If using this hint:
      - The User Agent <em class="rfc2119">MUST</em> respect other explicitly
        set encoding options when configuring the encoder, whether they are
        codec-specific encoding options or not.
      - The User Agent <em class="rfc2119">SHOULD</em> make a best-effort
        attempt to use additional configuration options to improve encoding
        quality, according to the goals defined by the corresponding
        [=video content hint=].

    NOTE: Some encoder options are implementation specific, and mappings between
        {{VideoEncoderConfig/contentHint}} and those options cannot be
        prescribed.

    The User Agent <em class="rfc2119">MUST NOT</em> refuse the configuration
    if it doesn't support this content hint.
    See {{VideoEncoder/isConfigSupported()}}.
  </dd>

</dl>

Hardware Acceleration{#hardware-acceleration}
---------------------------------------------
<xmp class='idl'>
enum HardwareAcceleration {
  "no-preference",
  "prefer-hardware",
  "prefer-software",
};
</xmp>

When supported, hardware acceleration offloads encoding or decoding to
specialized hardware. {{HardwareAcceleration/prefer-hardware}} and
{{HardwareAcceleration/prefer-software}} are hints. While User Agents
<em class="rfc2119">SHOULD</em> respect these values when possible, User
Agents may ignore these values in some or all circumstances for any reason.

To prevent fingerprinting, if a User Agent implements [[media-capabilities]],
the User Agent <em class="rfc2119">MUST</em> ensure rejection or acceptance of
a given {{HardwareAcceleration}} preference reveals no additional information
on top of what is inherent to the User Agent and revealed by
[[media-capabilities]]. If a User Agent does not implement
[[media-capabilities]] for reasons of fingerprinting, they
<em class="rfc2119">SHOULD</em> ignore the {{HardwareAcceleration}} preference.

<div class='note'>
  NOTE: Good examples of when a User Agent can ignore
  {{HardwareAcceleration/prefer-hardware}} or
  {{HardwareAcceleration/prefer-software}} are for reasons of user privacy or
  circumstances where the User Agent determines an alternative setting would
  better serve the end user.

  Most authors will be best served by using the default of
  {{HardwareAcceleration/no-preference}}. This gives the User Agent flexibility
  to optimize based on its knowledge of the system and configuration. A common
  strategy will be to prioritize hardware acceleration at higher resolutions
  with a fallback to software codecs if hardware acceleration fails.

  Authors are encouraged to carefully weigh the tradeoffs when setting a
  hardware acceleration preference. The precise tradeoffs will be
  device-specific, but authors can generally expect the following:

  * Setting a value of {{HardwareAcceleration/prefer-hardware}} or
      {{HardwareAcceleration/prefer-software}} can significantly restrict what
      configurations are supported. It can occur that the user's device does not
      offer acceleration for any codec, or only for the most common profiles of
      older codecs. It can also occur that a given User Agent lacks a software
      based codec implementation.
  * Hardware acceleration does not simply imply faster encoding / decoding.
      Hardware acceleration often has higher startup latency but more consistent
      throughput performance. Acceleration will generally reduce CPU load.
  * For decoding, hardware acceleration is often less robust to inputs that are
      mislabeled or violate the relevant codec specification.
  * Hardware acceleration will often be more power efficient than purely
      software based codecs.
  * For lower resolution content, the overhead added by hardware acceleration
      can yield decreased performance and power efficiency compared to purely
      software based codecs.

  Given these tradeoffs, a good example of using "prefer-hardware" would be if
  an author intends to provide their own software based fallback via
  WebAssembly.

  Alternatively, a good example of using "prefer-software" would be if an author
  is especially sensitive to the higher startup latency or decreased robustness
  generally associated with hardware acceleration.
</div>

<dl>
  <dt><dfn enum-value for=HardwareAcceleration>no-preference</dfn></dt>
  <dd>
    Indicates that the User Agent <em class="rfc2119">MAY</em> use hardware
    acceleration if it is available and compatible with other aspects of the
    codec configuration.
  </dd>
  <dt><dfn enum-value for=HardwareAcceleration>prefer-software</dfn></dt>
  <dd>
    Indicates that the User Agent <em class="rfc2119">SHOULD</em> prefer a
    software codec implementation.  User Agents may ignore this value for any
    reason.

    NOTE: This can cause the configuration to be unsupported on platforms where
    an unaccelerated codec is unavailable or is incompatible with other aspects
    of the codec configuration.
  </dd>
  <dt><dfn enum-value for=HardwareAcceleration>prefer-hardware</dfn></dt>
  <dd>
    Indicates that the User Agent <em class="rfc2119">SHOULD</em> prefer
    hardware acceleration. User Agents may ignore this value for any reason.

    NOTE: This can cause the configuration to be unsupported on platforms where
    an accelerated codec is unavailable or is incompatible with other aspects of
    the codec configuration.
  </dd>
</dl>

Alpha Option{#alpha-option}
---------------------------
<xmp class='idl'>
enum AlphaOption {
  "keep",
  "discard",
};
</xmp>

Describes how the user agent <em class="rfc2119">SHOULD</em> behave when dealing
with alpha channels, for a variety of different operations.

<dl>
  <dt><dfn enum-value for=AlphaOption>keep</dfn></dt>
  <dd>
    Indicates that the user agent <em class="rfc2119">SHOULD</em> preserve alpha
    channel data for {{VideoFrame}}s, if it is present.
  </dd>
  <dt><dfn enum-value for=AlphaOption>discard</dfn></dt>
  <dd>
    Indicates that the user agent <em class="rfc2119">SHOULD</em> ignore or
    remove {{VideoFrame}}'s alpha channel data.
  </dd>
</dl>

Latency Mode{#latency-mode}
---------------------------
<xmp class='idl'>
enum LatencyMode {
  "quality",
  "realtime"
};
</xmp>

: <dfn enum-value for=LatencyMode>quality</dfn>
:: Indicates that the User Agent <em class="rfc2119">SHOULD</em> optimize for
    encoding quality. In this mode:
    * User Agents <em class="rfc2119">MAY</em> increase encoding latency to
        improve quality.
    * User Agents <em class="rfc2119">MUST</em> not drop frames to achieve the
        target {{VideoEncoderConfig/bitrate}} and/or
        {{VideoEncoderConfig/framerate}}.
    * {{VideoEncoderConfig/framerate}} <em class="rfc2119">SHOULD</em> not be
        used as a target deadline for emitting encoded chunks.

: <dfn enum-value for=LatencyMode>realtime</dfn>
:: Indicates that the User Agent <em class="rfc2119">SHOULD</em> optimize for
    low latency. In this
    mode:
    * User Agents <em class="rfc2119">MAY</em> sacrifice quality to improve
        latency.
    * User Agents <em class="rfc2119">MAY</em> drop frames to achieve the target
        {{VideoEncoderConfig/bitrate}} and/or {{VideoEncoderConfig/framerate}}.
    * {{VideoEncoderConfig/framerate}} <em class="rfc2119">SHOULD</em> be used
        as a target deadline for emitting encoded chunks.

Configuration Equivalence{#config-equivalence}
----------------------------------------------
Two dictionaries are <dfn>equal dictionaries</dfn> if they contain the same
keys and values. For nested dictionaries, apply this definition recursively.


VideoEncoderEncodeOptions{#video-encoder-options}
-------------------------------------------------

<xmp class='idl'>
dictionary VideoEncoderEncodeOptions {
  boolean keyFrame = false;
};
</xmp>

NOTE: Codec-specific extensions to {{VideoEncoderEncodeOptions}} are described in
    their registrations in the [[WEBCODECS-CODEC-REGISTRY]].

<dl>
  <dt><dfn dict-member for=VideoEncoderEncodeOptions>keyFrame</dfn></dt>
  <dd>
    A value of `true` indicates that the given frame
    <em class="rfc2119">MUST</em> be encoded as a key frame. A value of `false`
    indicates that the User Agent has flexibility to decide whether the frame
    will be encoded as a [=key frame=].
  </dd>
</dl>


VideoEncoderBitrateMode{#video-encoder-bitrate-mode}
-------------------------------------------
<xmp class='idl'>
  enum VideoEncoderBitrateMode {
    "constant",
    "variable",
    "quantizer"
  };
</xmp>

<dl>
  <dt><dfn enum-value for=VideoEncoderBitrateMode>constant</dfn></dt>
  <dd>Encode at a constant bitrate. See {{VideoEncoderConfig/bitrate}}.</dd>
  <dt><dfn enum-value for=VideoEncoderBitrateMode>variable</dfn></dt>
  <dd>
    Encode using a variable bitrate, allowing more space to be used for
    complex signals and less space for less complex signals.
    See {{VideoEncoderConfig/bitrate}}.
  </dd>
  <dt><dfn enum-value for=VideoEncoderBitrateMode>quantizer</dfn></dt>
  <dd>
    Encode using a quantizer, that is specified for each video
    frame in codec specific extensions of {{VideoEncoderEncodeOptions}}.
  </dd>
</dl>



CodecState{#codec-state}
------------------------
<xmp class='idl'>
enum CodecState {
  "unconfigured",
  "configured",
  "closed"
};
</xmp>

<dl>
  <dt><dfn enum-value for=CodecState>unconfigured</dfn></dt>
  <dd>The codec is not configured for encoding or decoding.</dd>
  <dt><dfn enum-value for=CodecState>configured</dfn></dt>
  <dd>
    A valid configuration has been provided. The codec is ready for encoding or
        decoding.
  </dd>
  <dt><dfn enum-value for=CodecState>closed</dfn></dt>
  <dd>
    The codec is no longer usable and underlying [=system resources=] have
        been released.
  </dd>
</dl>

WebCodecsErrorCallback{#error-callback}
---------------------------------------
<xmp class='idl'>
callback WebCodecsErrorCallback = undefined(DOMException error);
</xmp>


Encoded Media Interfaces (Chunks) {#encoded-media-interfaces}
=============================================================
These interfaces represent chunks of encoded media.

EncodedAudioChunk Interface {#encodedaudiochunk-interface}
------------------------------------------------------------
<xmp class='idl'>
[Exposed=(Window,DedicatedWorker), Serializable]
interface EncodedAudioChunk {
  constructor(EncodedAudioChunkInit init);
  readonly attribute EncodedAudioChunkType type;
  readonly attribute long long timestamp;          // microseconds
  readonly attribute unsigned long long? duration; // microseconds
  readonly attribute unsigned long byteLength;

  undefined copyTo(AllowSharedBufferSource destination);
};

dictionary EncodedAudioChunkInit {
  required EncodedAudioChunkType type;
  [EnforceRange] required long long timestamp;    // microseconds
  [EnforceRange] unsigned long long duration;     // microseconds
  required AllowSharedBufferSource data;
  sequence<ArrayBuffer> transfer = [];
};

enum EncodedAudioChunkType {
    "key",
    "delta",
};
</xmp>

### Internal Slots ### {#encodedaudiochunk-internal-slots}
: <dfn attribute for=EncodedAudioChunk>[[internal data]]</dfn>
:: An array of bytes representing the encoded chunk data.
: <dfn attribute for=EncodedAudioChunk>\[[type]]</dfn>
:: Describes whether the chunk is a [=key chunk=].
: <dfn attribute for=EncodedAudioChunk>\[[timestamp]]</dfn>
:: The presentation timestamp, given in microseconds.
: <dfn attribute for=EncodedAudioChunk>\[[duration]]</dfn>
:: The presentation duration, given in microseconds.
: <dfn attribute for=EncodedAudioChunk>[[byte length]]</dfn>
:: The byte length of {{EncodedAudioChunk/[[internal data]]}}.

### Constructors ###{#encodedaudiochunk-constructors}
<dfn constructor for=EncodedAudioChunk title="EncodedAudioChunk(init)">
  EncodedAudioChunk(init)
</dfn>
1. If |init|.{{EncodedAudioChunkInit/transfer}} contains more than one reference
     to the same {{ArrayBuffer}}, then throw a {{DataCloneError}} {{DOMException}}.
2. For each |transferable| in |init|.{{EncodedAudioChunkInit/transfer}}:
     1. If {{platform object/[[Detached]]}} internal slot is `true`,
         then throw a {{DataCloneError}} {{DOMException}}.
3. Let |chunk| be a new {{EncodedAudioChunk}} object, initialized as follows
    1. Assign `init.type` to {{EncodedAudioChunk/[[type]]}}.
    2. Assign `init.timestamp` to {{EncodedAudioChunk/[[timestamp]]}}.
    3. If `init.duration` exists, assign it to
        {{EncodedAudioChunk/[[duration]]}}, or assign `null` otherwise.
    4. Assign `init.data.byteLength` to {{EncodedAudioChunk/[[byte length]]}};
    5. If |init|.{{EncodedAudioChunkInit/transfer}} contains an {{ArrayBuffer}}
        referenced by |init|.{{EncodedAudioChunkInit/data}} the User Agent
        <em class="rfc2119">MAY</em> choose to:
          1. Let |resource| be a new [=media resource=] referencing sample data
               in |init|.{{EncodedAudioChunkInit/data}}.
    6. Otherwise:
         1. Assign a copy of |init|.{{EncodedAudioChunkInit/data}}
              to {{EncodedAudioChunk/[[internal data]]}}.
4. For each |transferable| in |init|.{{EncodedAudioChunkInit/transfer}}:
    1. Perform [DetachArrayBuffer](https://tc39.es/ecma262/#sec-detacharraybuffer)
        on |transferable|
5. Return |chunk|.

### Attributes ###{#encodedaudiochunk-attributes}
: <dfn attribute for=EncodedAudioChunk>type</dfn>
:: Returns the value of {{EncodedAudioChunk/[[type]]}}.

: <dfn attribute for=EncodedAudioChunk>timestamp</dfn>
:: Returns the value of {{EncodedAudioChunk/[[timestamp]]}}.

: <dfn attribute for=EncodedAudioChunk>duration</dfn>
:: Returns the value of {{EncodedAudioChunk/[[duration]]}}.

: <dfn attribute for=EncodedAudioChunk>byteLength</dfn>
:: Returns the value of {{EncodedAudioChunk/[[byte length]]}}.

### Methods ###{#encodedaudiochunk-methods}
: <dfn method for=EncodedAudioChunk>copyTo(destination)</dfn>
:: When invoked, run these steps:
    1. If the {{EncodedAudioChunk/[[byte length]]}} of this {{EncodedAudioChunk}} is
        greater than in |destination|, throw a {{TypeError}}.
    2. Copy the {{EncodedAudioChunk/[[internal data]]}} into |destination|.

### Serialization ###{#encodedaudiochunk-serialization}
: The {{EncodedAudioChunk}} [=serialization steps=] (with |value|, |serialized|,
    and |forStorage|) are:
:: 1. If |forStorage| is `true`, throw a {{DataCloneError}}.
    2. For each {{EncodedAudioChunk}} internal slot in |value|, assign the value
        of each internal slot to a field in |serialized| with the same name as
        the internal slot.

: The {{EncodedAudioChunk}} [=deserialization steps=] (with |serialized| and
    |value|) are:
:: 1. For all named fields in |serialized|, assign the value of each named field
        to the {{EncodedAudioChunk}} internal slot in |value| with the same name
        as the named field.

NOTE: Since {{EncodedAudioChunk}}s are immutable, User
    Agents can choose to implement serialization using a reference counting
    model similar to [[#audiodata-transfer-serialization]].

EncodedVideoChunk Interface{#encodedvideochunk-interface}
-----------------------------------------------------------
<xmp class='idl'>
[Exposed=(Window,DedicatedWorker), Serializable]
interface EncodedVideoChunk {
  constructor(EncodedVideoChunkInit init);
  readonly attribute EncodedVideoChunkType type;
  readonly attribute long long timestamp;             // microseconds
  readonly attribute unsigned long long? duration;    // microseconds
  readonly attribute unsigned long byteLength;

  undefined copyTo(AllowSharedBufferSource destination);
};

dictionary EncodedVideoChunkInit {
  required EncodedVideoChunkType type;
  [EnforceRange] required long long timestamp;        // microseconds
  [EnforceRange] unsigned long long duration;         // microseconds
  required AllowSharedBufferSource data;
  sequence<ArrayBuffer> transfer = [];
};

enum EncodedVideoChunkType {
    "key",
    "delta",
};
</xmp>

### Internal Slots ### {#encodedvideochunk-internal-slots}
: <dfn attribute for=EncodedVideoChunk>[[internal data]]</dfn>
:: An array of bytes representing the encoded chunk data.
: <dfn attribute for=EncodedVideoChunk>\[[type]]</dfn>
:: The {{EncodedVideoChunkType}} of this {{EncodedVideoChunk}};
: <dfn attribute for=EncodedVideoChunk>\[[timestamp]]</dfn>
:: The presentation timestamp, given in microseconds.
: <dfn attribute for=EncodedVideoChunk>\[[duration]]</dfn>
:: The presentation duration, given in microseconds.
: <dfn attribute for=EncodedVideoChunk>[[byte length]]</dfn>
:: The byte length of {{EncodedVideoChunk/[[internal data]]}}.

### Constructors ###{#encodedvideochunk-constructors}
<dfn constructor for=EncodedVideoChunk title="EncodedVideoChunk(init)">
  EncodedVideoChunk(init)
</dfn>
1. If |init|.{{EncodedVideoChunkInit/transfer}} contains more than one reference
     to the same {{ArrayBuffer}}, then throw a {{DataCloneError}} {{DOMException}}.
2. For each |transferable| in |init|.{{EncodedVideoChunkInit/transfer}}:
     1. If {{platform object/[[Detached]]}} internal slot is `true`,
         then throw a {{DataCloneError}} {{DOMException}}.
3. Let |chunk| be a new {{EncodedVideoChunk}} object, initialized as follows
    1. Assign `init.type` to {{EncodedVideoChunk/[[type]]}}.
    2. Assign `init.timestamp` to {{EncodedVideoChunk/[[timestamp]]}}.
    3. If duration is present in init, assign `init.duration` to
        {{EncodedVideoChunk/[[duration]]}}. Otherwise, assign `null` to
        {{EncodedVideoChunk/[[duration]]}}.
    4. Assign `init.data.byteLength` to {{EncodedVideoChunk/[[byte length]]}};
    5. If |init|.{{EncodedVideoChunkInit/transfer}} contains an {{ArrayBuffer}}
        referenced by |init|.{{EncodedVideoChunkInit/data}} the User Agent
        <em class="rfc2119">MAY</em> choose to:
          1. Let |resource| be a new [=media resource=] referencing sample data
               in |init|.{{EncodedVideoChunkInit/data}}.
    6. Otherwise:
         1. Assign a copy of |init|.{{EncodedVideoChunkInit/data}}
              to {{EncodedVideoChunk/[[internal data]]}}.
4. For each |transferable| in |init|.{{EncodedVideoChunkInit/transfer}}:
    1. Perform [DetachArrayBuffer](https://tc39.es/ecma262/#sec-detacharraybuffer)
        on |transferable|
5. Return |chunk|.

### Attributes ###{#encodedvideochunk-attributes}
: <dfn attribute for=EncodedVideoChunk>type</dfn>
:: Returns the value of {{EncodedVideoChunk/[[type]]}}.

: <dfn attribute for=EncodedVideoChunk>timestamp</dfn>
:: Returns the value of {{EncodedVideoChunk/[[timestamp]]}}.

: <dfn attribute for=EncodedVideoChunk>duration</dfn>
:: Returns the value of {{EncodedVideoChunk/[[duration]]}}.

: <dfn attribute for=EncodedVideoChunk>byteLength</dfn>
:: Returns the value of {{EncodedVideoChunk/[[byte length]]}}.

### Methods ###{#encodedvideochunk-methods}
: <dfn method for=EncodedVideoChunk>copyTo(destination)</dfn>
:: When invoked, run these steps:
    1. If {{EncodedVideoChunk/[[byte length]]}} is greater than
        the {{EncodedVideoChunk/[[byte length]]}} of |destination|, throw a
        {{TypeError}}.
    2. Copy the {{EncodedVideoChunk/[[internal data]]}} into |destination|.

### Serialization ###{#encodedvideochunk-serialization}
: The {{EncodedVideoChunk}} [=serialization steps=] (with |value|, |serialized|,
    and |forStorage|) are:
:: 1. If |forStorage| is `true`, throw a {{DataCloneError}}.
    2. For each {{EncodedVideoChunk}} internal slot in |value|, assign the value
        of each internal slot to a field in |serialized| with the same name as
        the internal slot.

: The {{EncodedVideoChunk}} [=deserialization steps=] (with |serialized| and
    |value|) are:
:: 1. For all named fields in |serialized|, assign the value of each named field
        to the {{EncodedVideoChunk}} internal slot in |value| with the same name
        as the named field.

NOTE: Since {{EncodedVideoChunk}}s are immutable, User
    Agents can choose to implement serialization using a reference counting
    model similar to [[#videoframe-transfer-serialization]].


Raw Media Interfaces {#raw-media-interfaces}
====================================================
These interfaces represent unencoded (raw) media.

Memory Model {#raw-media-memory-model}
--------------------------------------

### Background ### {#raw-media-memory-model-background}

This section is non-normative.

Decoded media data <em class="rfc2119">MAY</em> occupy a large amount of system
memory. To minimize the need for expensive copies, this specification defines a
scheme for reference counting (`clone()` and `close()`).

NOTE: Authors are encouraged to call `close()` immediately when frames are
    no longer needed.

### Reference Counting ### {#raw-media-memory-model-reference-counting}

A <dfn>media resource</dfn> is storage for the actual pixel data or the audio
sample data described by a {{VideoFrame}} or {{AudioData}}.

The {{AudioData}} {{AudioData/[[resource reference]]}} and {{VideoFrame}}
{{VideoFrame/[[resource reference]]}} internal slots hold a reference to a
[=media resource=].

{{VideoFrame}}.{{VideoFrame/clone()}} and
{{AudioData}}.{{AudioData/clone()}} return new objects whose
`[[resource reference]]` points to the same [=media resource=] as the original
object.

{{VideoFrame}}.{{VideoFrame/close()}} and {{AudioData}}.{{AudioData/close()}}
will clear their `[[resource reference]]` slot, releasing the reference their
[=media resource=].

A [=media resource=] <em class="rfc2119">MUST</em> remain alive at least as long
as it continues to be referenced by a `[[resource reference]]`.

NOTE: When a [=media resource=] is no longer referenced by a
    `[[resource reference]]`, the resource can be destroyed. User Agents are
    encouraged to destroy such resources quickly to reduce memory pressure and
    facilitate resource reuse.

### Transfer and Serialization ### {#raw-media-serialization-and-transfer}

This section is non-normative.

{{AudioData}} and {{VideoFrame}} are both
[[HTML#transferable-objects|transferable]] and
[[HTML#serializable-objects|serializable]] objects. Their transfer and
serialization steps are defined in [[#audiodata-transfer-serialization]] and
[[#videoframe-transfer-serialization]] respectively.

Transferring an {{AudioData}} or {{VideoFrame}} moves its `[[resource
reference]]` to the destination object and closes (as in {{AudioData/close()}})
the source object. Authors <em class="rfc2119">MAY</em> use this facility
to move an {{AudioData}} or {{VideoFrame}} between realms without copying the
underlying [=media resource=].

Serializing an {{AudioData}} or {{VideoFrame}} effectively clones (as in
{{VideoFrame/clone()}}) the source object, resulting in two objects that
reference the same [=media resource=]. Authors <em class="rfc2119">MAY</em> use
this facility to clone an {{AudioData}} or {{VideoFrame}} to another realm
without copying the underlying [=media resource=].

AudioData Interface {#audiodata-interface}
---------------------------------------------

<xmp class='idl'>
[Exposed=(Window,DedicatedWorker), Serializable, Transferable]
interface AudioData {
  constructor(AudioDataInit init);

  readonly attribute AudioSampleFormat? format;
  readonly attribute float sampleRate;
  readonly attribute unsigned long numberOfFrames;
  readonly attribute unsigned long numberOfChannels;
  readonly attribute unsigned long long duration;  // microseconds
  readonly attribute long long timestamp;          // microseconds

  unsigned long allocationSize(AudioDataCopyToOptions options);
  undefined copyTo(AllowSharedBufferSource destination, AudioDataCopyToOptions options);
  AudioData clone();
  undefined close();
};

dictionary AudioDataInit {
  required AudioSampleFormat format;
  required float sampleRate;
  [EnforceRange] required unsigned long numberOfFrames;
  [EnforceRange] required unsigned long numberOfChannels;
  [EnforceRange] required long long timestamp;  // microseconds
  required BufferSource data;
  sequence<ArrayBuffer> transfer = [];
};
</xmp>

### Internal Slots ###{#audiodata-internal-slots}

: <dfn attribute for=AudioData>[[resource reference]]</dfn>
:: A reference to a [=media resource=] that stores the audio sample data for
    this {{AudioData}}.

: <dfn attribute for=AudioData>\[[format]]</dfn>
:: The {{AudioSampleFormat}} used by this {{AudioData}}. Will be `null` whenever
    the underlying format does not map to an {{AudioSampleFormat}} or when
    {{platform object/[[Detached]]}} is `true`.

: <dfn attribute for=AudioData>[[sample rate]]</dfn>
:: The sample-rate, in Hz, for this {{AudioData}}.

: <dfn attribute for=AudioData>[[number of frames]]</dfn>
:: The number of [=frames=] for this {{AudioData}}.

: <dfn attribute for=AudioData>[[number of channels]]</dfn>
:: The number of audio channels for this {{AudioData}}.

: <dfn attribute for=AudioData>\[[timestamp]]</dfn>
:: The presentation timestamp, in microseconds, for this {{AudioData}}.

### Constructors ###{#audiodata-constructors}
<dfn constructor for=AudioData title="AudioData(init)">
  AudioData(init)
</dfn>
1. If |init| is not a [=valid AudioDataInit=], throw a {{TypeError}}.
2. If |init|.{{AudioDataInit/transfer}} contains more than one reference
     to the same {{ArrayBuffer}}, then throw a {{DataCloneError}} {{DOMException}}.
3. For each |transferable| in |init|.{{AudioDataInit/transfer}}:
     1. If {{platform object/[[Detached]]}} internal slot is `true`,
         then throw a {{DataCloneError}} {{DOMException}}.
4. Let |frame| be a new {{AudioData}} object, initialized as follows:
    1. Assign `false` to {{platform object/[[Detached]]}}.
    2. Assign |init|.{{AudioDataInit/format}} to
        {{AudioData/[[format]]}}.
    3. Assign |init|.{{AudioDataInit/sampleRate}} to
        {{AudioData/[[sample rate]]}}.
    4. Assign |init|.{{AudioDataInit/numberOfFrames}} to
        {{AudioData/[[number of frames]]}}.
    5. Assign |init|.{{AudioDataInit/numberOfChannels}} to
        {{AudioData/[[number of channels]]}}.
    6. Assign |init|.{{AudioDataInit/timestamp}} to
        {{AudioData/[[timestamp]]}}.
    7. If |init|.{{AudioDataInit/transfer}} contains an {{ArrayBuffer}}
        referenced by |init|.{{AudioDataInit/data}} the User Agent
        <em class="rfc2119">MAY</em> choose to:
          1. Let |resource| be a new [=media resource=] referencing sample data
               in |data|.
    8. Otherwise:
        1. Let |resource| be a [=media resource=] containing a copy of
            |init|.{{AudioDataInit/data}}.
    9. Let |resourceReference| be a reference to |resource|.
    10. Assign |resourceReference| to {{AudioData/[[resource reference]]}}.
5. For each |transferable| in |init|.{{AudioDataInit/transfer}}:
    1. Perform [DetachArrayBuffer](https://tc39.es/ecma262/#sec-detacharraybuffer)
        on |transferable|
6. Return |frame|.

### Attributes ###{#audiodata-attributes}

: <dfn attribute for=AudioData>format</dfn>
:: The {{AudioSampleFormat}} used by this {{AudioData}}. Will be `null` whenever
    the underlying format does not map to a {{AudioSampleFormat}} or when
    {{platform object/[[Detached]]}} is `true`.

    The {{AudioData/format}} getter steps are to return
    {{AudioData/[[format]]}}.

: <dfn attribute for=AudioData>sampleRate</dfn>
:: The sample-rate, in Hz, for this {{AudioData}}.

    The {{AudioData/sampleRate}} getter steps are to return
    {{AudioData/[[sample rate]]}}.

: <dfn attribute for=AudioData>numberOfFrames</dfn>
:: The number of [=frames=] for this {{AudioData}}.

    The {{AudioData/numberOfFrames}} getter steps are to return
    {{AudioData/[[number of frames]]}}.

: <dfn attribute for=AudioData>numberOfChannels</dfn>
:: The number of audio channels for this {{AudioData}}.

    The {{AudioData/numberOfChannels}} getter steps are to return
    {{AudioData/[[number of channels]]}}.

: <dfn attribute for=AudioData>timestamp</dfn>
:: The presentation timestamp, in microseconds, for this {{AudioData}}.

    The {{AudioData/numberOfChannels}} getter steps are to return
    {{AudioData/[[timestamp]]}}.

: <dfn attribute for=AudioData>duration</dfn>
:: The duration, in microseconds, for this {{AudioData}}.

    The {{AudioData/duration}} getter steps are to:
    1. Let |microsecondsPerSecond| be `1,000,000`.
    2. Let |durationInSeconds| be the result of dividing
        {{AudioData/[[number of frames]]}} by {{AudioData/[[sample rate]]}}.
    3. Return the product of |durationInSeconds| and |microsecondsPerSecond|.

### Methods ###{#audiodata-methods}
: <dfn method for=AudioData>allocationSize(|options|)</dfn>
:: Returns the number of bytes required to hold the samples as described by
    |options|.

    When invoked, run these steps:
    1. If {{platform object/[[Detached]]}} is `true`, throw an
        {{InvalidStateError}} {{DOMException}}.
    2. Let |copyElementCount| be the result of running the
        [=Compute Copy Element Count=] algorithm with |options|.
    3. Let |destFormat| be the value of {{AudioData/[[format]]}}.
    4. If |options|.{{AudioDataCopyToOptions/format}} [=map/exists=], assign
        |options|.{{AudioDataCopyToOptions/format}} to |destFormat|.
    5. Let |bytesPerSample| be the number of bytes per sample, as defined by
        the |destFormat|.
    6. Return the product of multiplying |bytesPerSample| by
        |copyElementCount|.

: <dfn method for=AudioData>copyTo(|destination|, |options|)</dfn>
:: Copies the samples from the specified plane of the {{AudioData}} to the
    destination buffer.

    When invoked, run these steps:
    1. If {{platform object/[[Detached]]}} is `true`, throw an
        {{InvalidStateError}} {{DOMException}}.
    2. Let |copyElementCount| be the result of running the
        [=Compute Copy Element Count=] algorithm with |options|.
    3. Let |destFormat| be the value of {{AudioData/[[format]]}}.
    4. If |options|.{{AudioDataCopyToOptions/format}} [=map/exists=], assign
        |options|.{{AudioDataCopyToOptions/format}} to |destFormat|.
    5. Let |bytesPerSample| be the number of bytes per sample, as defined by
        the |destFormat|.
    6. If the product of multiplying |bytesPerSample| by |copyElementCount| is
        greater than `destination.byteLength`, throw a {{RangeError}}.
    7. Let |resource| be the [=media resource=] referenced by
        {{AudioData/[[resource reference]]}}.
    8. Let |planeFrames| be the region of |resource| corresponding to
        |options|.{{AudioDataCopyToOptions/planeIndex}}.
    9. Copy elements of |planeFrames| into |destination|, starting with the
        [=frame=] positioned at |options|.{{AudioDataCopyToOptions/frameOffset}}
        and stopping after |copyElementCount| samples have been copied. If
        |destFormat| does not equal {{AudioData/[[format]]}}, convert elements
        to the |destFormat| {{AudioSampleFormat}} while making the copy.

: <dfn method for=AudioData>clone()</dfn>
:: Creates a new AudioData with a reference to the same [=media resource=].

    When invoked, run these steps:
    1. If {{platform object/[[Detached]]}} is `true`, throw an
        {{InvalidStateError}} {{DOMException}}.
    2. Return the result of running the [=Clone AudioData=] algorithm with
        [=this=].

: <dfn method for=AudioData>close()</dfn>
:: Clears all state and releases the reference to the [=media resource=].
    Close is final.

    When invoked, run the [=Close AudioData=] algorithm with [=this=].

### Algorithms ### {#audiodata-algorithms}

: <dfn>Compute Copy Element Count</dfn> (with |options|)
:: Run these steps:
    1. Let |destFormat| be the value of {{AudioData/[[format]]}}.
    2. If |options|.{{AudioDataCopyToOptions/format}} [=map/exists=], assign
        |options|.{{AudioDataCopyToOptions/format}} to |destFormat|.
    3. If |destFormat| describes an [=interleaved=] {{AudioSampleFormat}} and
        |options|.{{AudioDataCopyToOptions/planeIndex}} is greater than `0`,
        throw a {{RangeError}}.
    4. Otherwise, if |destFormat| describes a [=planar=] {{AudioSampleFormat}}
        and if |options|.{{AudioDataCopyToOptions/planeIndex}} is greater or
        equal to {{AudioData/[[number of channels]]}}, throw a
        {{RangeError}}.
    5. If {{AudioData/[[format]]}} does not equal |destFormat| and the User
        Agent does not support the requested {{AudioSampleFormat}} conversion,
        throw a {{NotSupportedError}} {{DOMException}}.
        Conversion to {{f32-planar}} <em class="rfc2119">MUST</em> always be
        supported.
    6. Let |frameCount| be the number of frames in the plane identified by
        |options|.{{AudioDataCopyToOptions/planeIndex}}.
    7. If |options|.{{AudioDataCopyToOptions/frameOffset}} is greater than or
        equal to |frameCount|, throw a {{RangeError}}.
    8. Let |copyFrameCount| be the difference of subtracting
        |options|.{{AudioDataCopyToOptions/frameOffset}} from |frameCount|.
    9. If |options|.{{AudioDataCopyToOptions/frameCount}} [=map/exists=]:
        1. If |options|.{{AudioDataCopyToOptions/frameCount}} is greater than
            |copyFrameCount|, throw a {{RangeError}}.
        2. Otherwise, assign |options|.{{AudioDataCopyToOptions/frameCount}}
            to |copyFrameCount|.
    10. Let |elementCount| be |copyFrameCount|.
    11. If |destFormat| describes an [=interleaved=]
        {{AudioSampleFormat}}, multiply |elementCount| by
        {{AudioData/[[number of channels]]}}
    12. return |elementCount|.

: <dfn>Clone AudioData</dfn> (with |data|)
:: Run these steps:
    1. Let |clone| be a new {{AudioData}} initialized as follows:
        1. Let |resource| be the [=media resource=] referenced by |data|'s
            {{AudioData/[[resource reference]]}}.
        2. Let |reference| be a new reference to |resource|.
        3. Assign |reference| to {{AudioData/[[resource reference]]}}.
        4. Assign the values of |data|'s {{platform object/[[Detached]]}},
            {{AudioData/[[format]]}}, {{AudioData/[[sample rate]]}},
            {{AudioData/[[number of frames]]}},
            {{AudioData/[[number of channels]]}}, and
            {{AudioData/[[timestamp]]}} slots to the corresponding slots in
            |clone|.
    2. Return |clone|.


: <dfn>Close AudioData</dfn> (with |data|)
:: Run these steps:
    1. Assign `true` to |data|'s {{platform object/[[Detached]]}} internal slot.
    2. Assign `null` to |data|'s {{AudioData/[[resource reference]]}}.
    3. Assign `0` to |data|'s {{AudioData/[[sample rate]]}}.
    4. Assign `0` to |data|'s {{AudioData/[[number of frames]]}}.
    5. Assign `0` to |data|'s {{AudioData/[[number of channels]]}}.
    6. Assign `null` to |data|'s {{AudioData/[[format]]}}.

: To check if a {{AudioDataInit}} is a
    <dfn>valid AudioDataInit</dfn>, run these steps:
:: 1. If {{AudioDataInit/sampleRate}} less than or equal to `0`, return `false`.
    2. If {{AudioDataInit/numberOfFrames}} = `0`, return `false`.
    3. If {{AudioDataInit/numberOfChannels}} = `0`, return `false`.
    4. Verify {{AudioDataInit/data}} has enough data by running the following
        steps:
        1. Let |totalSamples| be the product of multiplying
            {{AudioDataInit/numberOfFrames}} by
            {{AudioDataInit/numberOfChannels}}.
        2. Let |bytesPerSample| be the number of bytes per sample, as defined by
            the {{AudioDataInit/format}}.
        3. Let |totalSize| be the product of multiplying |bytesPerSample| with
            |totalSamples|.
        4. Let |dataSize| be the size in bytes of {{AudioDataInit/data}}.
        5. If |dataSize| is less than |totalSize|, return false.
    5. Return `true`.

<div class='note'>
Note: It's expected that {{AudioDataInit}}'s {{AudioDataInit/data}}'s memory
    layout matches the expectations of the [=planar=] or [=interleaved=]
    {{AudioDataInit/format}}. There is no real way to verify whether the samples
    conform to their {{AudioSampleFormat}}.
</div>

### Transfer and Serialization ###{#audiodata-transfer-serialization}

: The {{AudioData}} [=transfer steps=] (with |value| and |dataHolder|) are:
:: 1. If |value|'s {{platform object/[[Detached]]}} is `true`, throw a
        {{DataCloneError}} {{DOMException}}.
    2. For all {{AudioData}} internal slots in |value|, assign the value of
        each internal slot to a field in |dataHolder| with the same name as the
        internal slot.
    3. Run the [=Close AudioData=] algorithm with |value|.

: The {{AudioData}} [=transfer-receiving steps=] (with |dataHolder| and |value|)
    are:
:: 1. For all named fields in |dataHolder|, assign the value of each named
        field to the {{AudioData}} internal slot in |value| with the same name
        as the named field.

: The {{AudioData}} [=serialization steps=] (with |value|, |serialized|, and
    |forStorage|) are:
:: 1. If |value|'s {{platform object/[[Detached]]}} is `true`, throw a
        {{DataCloneError}} {{DOMException}}.
    2. If |forStorage| is `true`, throw a {{DataCloneError}}.
    3. Let |resource| be the [=media resource=] referenced by
            |value|'s {{AudioData/[[resource reference]]}}.
    4. Let |newReference| be a new reference to |resource|.
    5. Assign |newReference| to |serialized.resource reference|.
    6. For all remaining {{AudioData}} internal slots (excluding
        {{AudioData/[[resource reference]]}}) in |value|, assign the value of
        each internal slot to a field in |serialized| with the same name as the
        internal slot.

: The {{AudioData}} [=deserialization steps=] (with |serialized| and |value|)
    are:
:: 1. For all named fields in |serialized|, assign the value of each named
        field to the {{AudioData}} internal slot in |value| with the same name
        as the named field.

### AudioDataCopyToOptions ### {#audiodata-copy-to-options}

<xmp class='idl'>
dictionary AudioDataCopyToOptions {
  [EnforceRange] required unsigned long planeIndex;
  [EnforceRange] unsigned long frameOffset = 0;
  [EnforceRange] unsigned long frameCount;
  AudioSampleFormat format;
};
</xmp>

: <dfn dict-member for=AudioDataCopyToOptions>planeIndex</dfn>
:: The index identifying the plane to copy from.

: <dfn dict-member for=AudioDataCopyToOptions>frameOffset</dfn>
:: An offset into the source plane data indicating which [=frame=] to begin
    copying from. Defaults to `0`.

: <dfn dict-member for=AudioDataCopyToOptions>frameCount</dfn>
:: The number of [=frames=] to copy. If not provided, the copy will include all
    [=frames=] in the plane beginning with {{AudioDataCopyToOptions/frameOffset}}.

: <dfn dict-member for=AudioDataCopyToOptions>format</dfn>
:: The output {{AudioSampleFormat}} for the destination data. If not provided,
    the resulting copy will use [=this=] AudioData's {{AudioData/[[format]]}}.
    Invoking {{AudioData/copyTo()}} will throw a {{NotSupportedError}} if
    conversion to the requested format is not supported. Conversion from any
    {{AudioSampleFormat}} to {{f32-planar}} <em class="rfc2119">MUST</em> always
    be supported.

    NOTE: Authors seeking to integrate with [[WEBAUDIO]] can request
        {{f32-planar}} and use the resulting copy to create and {{AudioBuffer}}
        or render via {{AudioWorklet}}.

## Audio Sample Format ##{#audio-sample-formats}

An audio sample format describes the numeric type used to represent a
single sample (e.g. 32-bit floating point) and the arrangement of samples from
different channels as either [=interleaved=] or [=planar=]. The <dfn>audio
sample type</dfn> refers solely to the numeric type and interval used to store
the data, this is {{u8}}, {{s16}}, {{s32}}, or {{f32}} for respectively
unsigned 8-bits, signed 16-bits, signed 32-bits, and 32-bits
floating point number. The [[#audio-buffer-arrangement|audio buffer
arrangement]] refers solely to the way the samples are laid out in memory
([=planar=] or [=interleaved=]).

A <dfn>sample</dfn> refers to a single value that is the magnitude of a
signal at a particular point in time in a particular channel.

A <dfn>frame</dfn> or (sample-frame) refers to a set of values of all channels
of a multi-channel signal, that happen at the exact same time.

NOTE: Consequently, if an audio signal is mono (has only one channel), a frame
and a sample refer to the same thing.

All audio [=samples=] in this specification are using linear pulse-code
modulation (Linear PCM): quantization levels are uniform between values.

NOTE: The Web Audio API, that is expected to be used with this specification,
also uses Linear PCM.

<xmp class='idl'>
enum AudioSampleFormat {
  "u8",
  "s16",
  "s32",
  "f32",
  "u8-planar",
  "s16-planar",
  "s32-planar",
  "f32-planar",
};
</xmp>

: <dfn enum-value for=AudioSampleFormat>u8</dfn>
:: [[WEBIDL#idl-octet|8-bit unsigned integer]] [=samples=] with [=interleaved=] [[#audio-buffer-arrangement|channel arrangement]].

: <dfn enum-value for=AudioSampleFormat>s16</dfn>
:: [[WEBIDL#idl-short|16-bit signed integer]] [=samples=] with [=interleaved=] [[#audio-buffer-arrangement|channel arrangement]].

: <dfn enum-value for=AudioSampleFormat>s32</dfn>
:: [[WEBIDL#idl-long|32-bit signed integer]] [=samples=] with [=interleaved=] [[#audio-buffer-arrangement|channel arrangement]].

: <dfn enum-value for=AudioSampleFormat>f32</dfn>
:: [[WEBIDL#idl-float|32-bit float]] [=samples=] with [=interleaved=] [[#audio-buffer-arrangement|channel arrangement]].

: <dfn enum-value for=AudioSampleFormat>u8-planar</dfn>
:: [[WEBIDL#idl-octet|8-bit unsigned integer]] [=samples=] with [=planar=] [[#audio-buffer-arrangement|channel arrangement]].

: <dfn enum-value for=AudioSampleFormat>s16-planar</dfn>
:: [[WEBIDL#idl-short|16-bit signed integer]] [=samples=] with [=planar=] [[#audio-buffer-arrangement|channel arrangement]].

: <dfn enum-value for=AudioSampleFormat>s32-planar</dfn>
:: [[WEBIDL#idl-long|32-bit signed integer]] [=samples=] with [=planar=] [[#audio-buffer-arrangement|channel arrangement]].

: <dfn enum-value for=AudioSampleFormat>f32-planar</dfn>
:: [[WEBIDL#idl-float|32-bit float]] [=samples=] with [=planar=] [[#audio-buffer-arrangement|channel arrangement]].

### Arrangement of audio buffer ### {#audio-buffer-arrangement}

When an {{AudioData}} has an {{AudioSampleFormat}} that is
<dfn>interleaved</dfn>, the audio samples from different channels are laid out
consecutively in the same buffer, in the order described in the section
[[#audio-channel-ordering]]. The {{AudioData}} has a single plane, that contains a
number of elements therefore equal to {{AudioData/[[number of frames]]}} *
{{AudioData/[[number of channels]]}}.

When an {{AudioData}} has an {{AudioSampleFormat}} that is
<dfn>planar</dfn>, the audio samples from different channels are laid out
in different buffers, themselves arranged in an order described in the section
[[#audio-channel-ordering]]. The {{AudioData}} has a number of planes equal to the
{{AudioData}}'s {{AudioData/[[number of channels]]}}. Each plane contains
{{AudioData/[[number of frames]]}} elements.

NOTE: The [[WEBAUDIO|Web Audio API]] currently uses {{f32-planar}} exclusively.

<div class='note'>
NOTE: The following diagram exemplifies the memory layout of [=planar=] versus
    [=interleaved=] {{AudioSampleFormat}}s

<img alt="Graphical representation the memory layout of interleaved and planar
    formats" src="images/planar_interleaved.svg" width="735" height="455">
</div>

### Magnitude of the audio samples ### {#audio-samples-magnitude}

The <dfn>minimum value</dfn> and <dfn>maximum value</dfn> of an audio sample,
for a particular audio sample type, are the values below which
(respectively above which) audio clipping might occur. They are otherwise regular
types, that can hold values outside this interval during intermediate
processing.

The <dfn>bias value</dfn> for an audio sample type is the value that often
corresponds to the middle of the range (but often the range is not symmetrical).
An audio buffer comprised only of values equal to the [=bias value=] is silent.

<table id="sample-types">
<thead>
<tr class="header">
<th>[=Audio sample type|Sample type=]</th>
<th>IDL type</th>
<th>[=Minimum value=]</th>
<th>[=Bias value=]</th>
<th>[=Maximum value=]</th>
</tr>
</thead>
<tbody>
<tr class="odd">
<td>{{u8}}</td>
<td>[[WEBIDL#idl-octet|octet]]</td>
<td>0</td>
<td>128</td>
<td>+255</td>
</tr>
<tr class="even">
<td>{{s16}}</td>
<td>[[WEBIDL#idl-short|short]]</td>
<td>-32768</td>
<td>0</td>
<td>+32767</td>
</tr>
<tr class="odd">
<td>{{s32}}</td>
<td>[[WEBIDL#idl-long|long]]</td>
<td>-2147483648</td>
<td>0</td>
<td>+2147483647</td>
</tr>
<tr class="even">
<td>{{f32}}</td>
<td>[[WEBIDL#idl-float|float]]</td>
<td>-1.0</td>
<td>0.0</td>
<td>+1.0</td>
</tr>
</tbody>
</table>

NOTE: There is no data type that can hold 24 bits of information conveniently,
but audio content using 24-bit samples is common, so 32-bits integers are
commonly used to hold 24-bit content.

{{AudioData}} containing 24-bit samples <em class="rfc2119">SHOULD</em> store those
samples in {{s32}} or {{f32}}. When samples are stored in {{s32}}, each sample
<em class="rfc2119">MUST</em> be left-shifted by `8` bits. By virtue of this
process, samples outside of the valid 24-bit range ([-8388608, +8388607]) will
be clipped. To avoid clipping and ensure lossless transport, samples
<em class="rfc2119">MAY</em> be converted to {{f32}}.

NOTE: While clipping is unavoidable in {{u8}}, {{s16}}, and {{s32}} samples due
to their storage types, implementations <em class="rfc2119">SHOULD</em> take
care not to clip internally when handling {{f32}} samples.

### Audio channel ordering ### {#audio-channel-ordering}

When decoding, the ordering of the audio channels in the resulting {{AudioData}}
<em class="rfc2119">MUST</em> be the same as what is present in the
{{EncodedAudioChunk}}.

When encoding, the ordering of the audio channels in the resulting
{{EncodedAudioChunk}} <em class="rfc2119">MUST</em> be the same as what is
preset in the given {{AudioData}}.

In other terms, no channel reordering is performed when encoding and decoding.

NOTE: The container either implies or specifies the channel mapping: the
channel attributed to a particular channel index.


VideoFrame Interface {#videoframe-interface}
--------------------------------------------

NOTE: {{VideoFrame}} is a {{CanvasImageSource}}. A {{VideoFrame}} can be
    passed to any method accepting a {{CanvasImageSource}}, including
    {{CanvasDrawImage}}'s {{CanvasDrawImage/drawImage()}}.

<xmp class='idl'>
[Exposed=(Window,DedicatedWorker), Serializable, Transferable]
interface VideoFrame {
  constructor(CanvasImageSource image, optional VideoFrameInit init = {});
  constructor(AllowSharedBufferSource data, VideoFrameBufferInit init);

  readonly attribute VideoPixelFormat? format;
  readonly attribute unsigned long codedWidth;
  readonly attribute unsigned long codedHeight;
  readonly attribute DOMRectReadOnly? codedRect;
  readonly attribute DOMRectReadOnly? visibleRect;
  readonly attribute double rotation;
  readonly attribute boolean flip;
  readonly attribute unsigned long displayWidth;
  readonly attribute unsigned long displayHeight;
  readonly attribute unsigned long long? duration;  // microseconds
  readonly attribute long long timestamp;           // microseconds
  readonly attribute VideoColorSpace colorSpace;

  VideoFrameMetadata metadata();

  unsigned long allocationSize(
      optional VideoFrameCopyToOptions options = {});
  Promise<sequence<PlaneLayout>> copyTo(
      AllowSharedBufferSource destination,
      optional VideoFrameCopyToOptions options = {});
  VideoFrame clone();
  undefined close();
};

dictionary VideoFrameInit {
  unsigned long long duration;  // microseconds
  long long timestamp;          // microseconds
  AlphaOption alpha = "keep";

  // Default matches image. May be used to efficiently crop. Will trigger
  // new computation of displayWidth and displayHeight using image's pixel
  // aspect ratio unless an explicit displayWidth and displayHeight are given.
  DOMRectInit visibleRect;

  double rotation = 0;
  boolean flip = false;

  // Default matches image unless visibleRect is provided.
  [EnforceRange] unsigned long displayWidth;
  [EnforceRange] unsigned long displayHeight;

  VideoFrameMetadata metadata;
};

dictionary VideoFrameBufferInit {
  required VideoPixelFormat format;
  required [EnforceRange] unsigned long codedWidth;
  required [EnforceRange] unsigned long codedHeight;
  required [EnforceRange] long long timestamp;  // microseconds
  [EnforceRange] unsigned long long duration;  // microseconds

  // Default layout is tightly-packed.
  sequence<PlaneLayout> layout;

  // Default visible rect is coded size positioned at (0,0)
  DOMRectInit visibleRect;

  double rotation = 0;
  boolean flip = false;

  // Default display dimensions match visibleRect.
  [EnforceRange] unsigned long displayWidth;
  [EnforceRange] unsigned long displayHeight;

  VideoColorSpaceInit colorSpace;

  sequence<ArrayBuffer> transfer = [];

  VideoFrameMetadata metadata;
};

dictionary VideoFrameMetadata {
  // Possible members are recorded in the VideoFrame Metadata Registry.
};
</xmp>

### Internal Slots ###{#videoframe-internal-slots}

: <dfn attribute for=VideoFrame>[[resource reference]]</dfn>
:: A reference to the [=media resource=] that stores the pixel data for
    this frame.

: <dfn attribute for=VideoFrame>\[[format]]</dfn>
:: A {{VideoPixelFormat}} describing the pixel format of the {{VideoFrame}}.
    Will be `null` whenever the underlying format does not map to a
    {{VideoPixelFormat}} or when {{platform object/[[Detached]]}} is `true`.

: <dfn attribute for=VideoFrame>[[coded width]]</dfn>
:: Width of the {{VideoFrame}} in pixels, potentially including non-visible
    padding, and prior to considering potential ratio adjustments.

: <dfn attribute for=VideoFrame>[[coded height]]</dfn>
:: Height of the {{VideoFrame}} in pixels, potentially including non-visible
    padding, and prior to considering potential ratio adjustments.

: <dfn attribute for=VideoFrame>[[visible left]]</dfn>
:: The number of pixels defining the left offset of the visible rectangle.

: <dfn attribute for=VideoFrame>[[visible top]]</dfn>
:: The number of pixels defining the top offset of the visible rectangle.

: <dfn attribute for=VideoFrame>[[visible width]]</dfn>
:: The width of pixels to include in visible rectangle, starting from
    {{VideoFrame/[[visible left]]}}.

: <dfn attribute for=VideoFrame>[[visible height]]</dfn>
:: The height of pixels to include in visible rectangle, starting from
    {{VideoFrame/[[visible top]]}}.

: <dfn attribute for=VideoFrame>\[[rotation]]</dfn>
:: The rotation to applied to the {{VideoFrame}} when rendered, in degrees
    clockwise. Rotation applies before flip.

: <dfn attribute for=VideoFrame>\[[flip]]</dfn>
:: Whether a horizontal flip is applied to the {{VideoFrame}} when rendered.
    Flip is applied after rotation.

: <dfn attribute for=VideoFrame>[[display width]]</dfn>
:: Width of the {{VideoFrame}} when displayed after applying aspect ratio
    adjustments.

: <dfn attribute for=VideoFrame>[[display height]]</dfn>
:: Height of the {{VideoFrame}} when displayed after applying aspect ratio
    adjustments.

: <dfn attribute for=VideoFrame>\[[duration]]</dfn>
:: The presentation duration, given in microseconds. The duration is copied
        from the {{EncodedVideoChunk}} corresponding to this {{VideoFrame}}.

: <dfn attribute for=VideoFrame>\[[timestamp]]</dfn>
::  The presentation timestamp, given in microseconds. The timestamp is copied
    from the {{EncodedVideoChunk}} corresponding to this {{VideoFrame}}.

: <dfn attribute for=VideoFrame>[[color space]]</dfn>
:: The {{VideoColorSpace}} associated with this frame.

: <dfn attribute for=VideoFrame>\[[metadata]]</dfn>
:: The {{VideoFrameMetadata}} associated with this frame.
    Possible members are recorded in [[webcodecs-video-frame-metadata-registry]].
    By design, all {{VideoFrameMetadata}} properties are serializable.

### Constructors ###{#videoframe-constructors}

<dfn constructor for=VideoFrame title="VideoFrame(image, init)">
  VideoFrame(image, init)
</dfn>
1. [=Canvas/Check the usability of the image argument=]. If this throws an
    exception or returns <var ignore=''>bad</var>, then throw an
    {{InvalidStateError}} {{DOMException}}.
2. If |image| [=Canvas/is not origin-clean=], then throw a {{SecurityError}} {{DOMException}}.
3. Let |frame| be a new {{VideoFrame}}.
5. Switch on |image|:

    NOTE: Authors are encouraged to provide a meaningful timestamp unless it is
        implicitly provided by the {{CanvasImageSource}} at construction.
        Interfaces that consume {{VideoFrame}}s can rely on this value for
        timing decisions. For example, {{VideoEncoder}} can use
        {{VideoFrame/timestamp}} values to guide rate control (see
        {{VideoEncoderConfig/framerate}}).

    - {{HTMLImageElement}}
    - {{SVGImageElement}}
        1. If {{VideoFrameInit/timestamp}} does not [=map/exist=] in
            |init|, throw a {{TypeError}}.
        2. If |image|'s media data has no [=natural dimensions=]
            (e.g., it's a vector graphic with no specified content size), then
            throw an {{InvalidStateError}} {{DOMException}}.
        3. Let |resource| be a new [=media resource=] containing a copy of
            |image|'s media data. If this is an animated image, |image|'s
            [=bitmap data=] <em class="rfc2119">MUST</em> only be taken from the
            default image of the animation (the one that the format defines is
            to be used when animation is not supported or is disabled), or, if
            there is no such image, the first frame of the animation.
        4. Let |codedWidth| and |codedHeight| be the width and height of
            |resource|.
        5. Let |baseRotation| and |baseFlip| describe the rotation and flip of
            |image| relative to |resource|.
        6. Let |defaultDisplayWidth| and |defaultDisplayHeight| be the
            [=natural width=] and [=natural height=] of |image|.
        7. Run the [=VideoFrame/Initialize Frame With Resource=]
            algorithm with |init|, |frame|, |resource|, |codedWidth|,
            |codedHeight|, |baseRotation|, |baseFlip|, |defaultDisplayWidth|,
            and |defaultDisplayHeight|.

    - {{HTMLVideoElement}}
        1. If |image|'s {{HTMLMediaElement/networkState}} attribute is
            {{HTMLMediaElement/NETWORK_EMPTY}}, then throw an
            {{InvalidStateError}} {{DOMException}}.
        2. Let |currentPlaybackFrame| be the {{VideoFrame}} at the [=current
            playback position=].
        3. If {{VideoFrameInit/metadata}} does not [=map/exist=] in |init|,
            assign |currentPlaybackFrame|.{{VideoFrame/[[metadata]]}} to it.
        4. Run the [=VideoFrame/Initialize Frame From Other Frame=] algorithm
            with |init|, |frame|, and |currentPlaybackFrame|.

    - {{HTMLCanvasElement}}
    - {{ImageBitmap}}
    - {{OffscreenCanvas}}
        1. If {{VideoFrameInit/timestamp}} does not [=map/exist=] in
            |init|, throw a {{TypeError}}.
        2. Let |resource| be a new [=media resource=] containing a copy of
            |image|'s [=bitmap data=].

            NOTE: Implementers are encouraged to avoid a deep copy by using
                reference counting where feasible.

        3. Let |width| be `image.width` and |height| be `image.height`.
        4. Run the [=VideoFrame/Initialize Frame With Resource=]
            algorithm with |init|, |frame|, |resource|, |width|, |height|,
            `0`, `false`, |width|, and |height|.

    - {{VideoFrame}}
        1. Run the [=VideoFrame/Initialize Frame From Other Frame=] algorithm
            with |init|, |frame|, and |image|.

6. Return |frame|.


<dfn constructor for=VideoFrame title="VideoFrame(data, init)">
  VideoFrame(data, init)
</dfn>
1. If |init| is not a [=valid VideoFrameBufferInit=], throw a {{TypeError}}.
2. Let |defaultRect| be «[ "x:" → `0`, "y" → `0`,
    "width" → |init|.{{VideoFrameBufferInit/codedWidth}}, "height" →
        |init|.{{VideoFrameBufferInit/codedWidth}} ]».
3. Let |overrideRect| be `undefined`.
4. If |init|.{{VideoFrameBufferInit/visibleRect}} [=map/exists=], assign its
    value to |overrideRect|.
5. Let |parsedRect| be the result of running the [=VideoFrame/Parse Visible
        Rect=] algorithm with |defaultRect|, |overrideRect|,
        |init|.{{VideoFrameBufferInit/codedWidth}},
        |init|.{{VideoFrameBufferInit/codedHeight}}, and
        |init|.{{VideoFrameBufferInit/format}}.
6. If |parsedRect| is an exception, return |parsedRect|.
7. Let |optLayout| be `undefined`.
8. If |init|.{{VideoFrameBufferInit/layout}} [=map/exists=], assign its value
    to |optLayout|.
9. Let |combinedLayout| be the result of running the [=VideoFrame/Compute
    Layout and Allocation Size=] algorithm with |parsedRect|,
    |init|.{{VideoFrameBufferInit/format}}, and |optLayout|.
10. If |combinedLayout| is an exception, throw |combinedLayout|.
11. If `data.byteLength` is less than |combinedLayout|’s
    [=combined buffer layout/allocationSize=], throw a {{TypeError}}.
12. If |init|.{{VideoFrameBufferInit/transfer}} contains more than one reference
     to the same {{ArrayBuffer}}, then throw a {{DataCloneError}} {{DOMException}}.
13. For each |transferable| in |init|.{{VideoFrameBufferInit/transfer}}:
     1. If {{platform object/[[Detached]]}} internal slot is `true`,
         then throw a {{DataCloneError}} {{DOMException}}.
14. If |init|.{{VideoFrameBufferInit/transfer}} contains an {{ArrayBuffer}}
    referenced by |data| the User Agent <em class="rfc2119">MAY</em> choose to:
    1. Let |resource| be a new [=media resource=] referencing pixel data in
        |data|.
15. Otherwise:
    1. Let |resource| be a new [=media resource=] containing a copy of |data|.
        Use {{VideoFrameBufferInit/visibleRect}} and {{VideoFrameBufferInit/layout}}
        to determine where in |data| the pixels for each plane reside.

        The User Agent <em class="rfc2119">MAY</em> choose to allocate
        |resource| with a larger coded size and plane strides to improve memory
        alignment. Increases will be reflected by {{VideoFrame/codedWidth}} and
        {{VideoFrame/codedHeight}}.  Additionally, the User Agent <em
        class="rfc2119">MAY</em> use {{VideoFrameBufferInit/visibleRect}} to
        copy only the visible rectangle.  It <em class="rfc2119">MAY</em> also
        reposition the visible rectangle within |resource|. The final position
        will be reflected by {{VideoFrame/visibleRect}}.

16. For each |transferable| in |init|.{{VideoFrameBufferInit/transfer}}:
    1. Perform [DetachArrayBuffer](https://tc39.es/ecma262/#sec-detacharraybuffer)
        on |transferable|
17. Let |resourceCodedWidth| be the coded width of |resource|.
18. Let |resourceCodedHeight| be the coded height of |resource|.
19. Let |resourceVisibleLeft| be the left offset for the visible rectangle of
    |resource|.
20. Let |resourceVisibleTop| be the top offset for the visible rectangle of
    |resource|.

    ISSUE: The spec <em class="rfc2119">SHOULD</em> provide definitions (and
      possibly diagrams) for coded size, visible rectangle, and display size.
      See [#166](https://github.com/w3c/webcodecs/issues/166).

21. Let |frame| be a new {{VideoFrame}} object initialized as follows:
    1. Assign |resourceCodedWidth|, |resourceCodedHeight|,
        |resourceVisibleLeft|, and |resourceVisibleTop| to
        {{VideoFrame/[[coded width]]}}, {{VideoFrame/[[coded height]]}},
        {{VideoFrame/[[visible left]]}}, and {{VideoFrame/[[visible top]]}}
        respectively.
    2. If |init|.{{VideoFrameBufferInit/visibleRect}} [=map/exists=]:
        1. Let |truncatedVisibleWidth| be the value of
            {{VideoFrameBufferInit/visibleRect}}.{{DOMRectInit/width}} after
            truncating.
        2. Assign |truncatedVisibleWidth| to {{VideoFrame/[[visible width]]}}.
        3. Let |truncatedVisibleHeight| be the value of
            {{VideoFrameBufferInit/visibleRect}}.{{DOMRectInit/height}} after
            truncating.
        4. Assign |truncatedVisibleHeight| to {{VideoFrame/[[visible height]]}}.
    3. Otherwise:
        1. Assign {{VideoFrame/[[coded width]]}} to
            {{VideoFrame/[[visible width]]}}.
        2. Assign {{VideoFrame/[[coded height]]}} to
            {{VideoFrame/[[visible height]]}}.
    4. Assign the result of running the [=VideoFrame/Parse Rotation=] algorithm,
        with |init|.{{VideoFrameBufferInit/rotation}}, to
        {{VideoFrame/[[rotation]]}}.
    5. Assign |init|.{{VideoFrameBufferInit/flip}} to
        {{VideoFrame/[[flip]]}}.
    6. If {{VideoFrameBufferInit/displayWidth}} and
        {{VideoFrameBufferInit/displayHeight}} [=map/exist=] in |init|, assign
        them to {{VideoFrame/[[display width]]}} and
        {{VideoFrame/[[display height]]}} respectively.
    7. Otherwise:
        1. If {{VideoFrame/[[rotation]]}} is equal to `0` or `180`:
            1. Assign {{VideoFrame/[[visible width]]}} to
                {{VideoFrame/[[display width]]}}.
            2. Assign {{VideoFrame/[[visible height]]}} to
                {{VideoFrame/[[display height]]}}.
        2. Otherwise:
            1. Assign {{VideoFrame/[[visible height]]}} to
                {{VideoFrame/[[display width]]}}.
            2. Assign {{VideoFrame/[[visible width]]}} to
                {{VideoFrame/[[display height]]}}.
    8. Assign |init|'s {{VideoFrameBufferInit/timestamp}} and
        {{VideoFrameBufferInit/duration}} to {{VideoFrame/[[timestamp]]}} and
        {{VideoFrame/[[duration]]}} respectively.
    9. Let |colorSpace| be `undefined`.
    10. If |init|.{{VideoFrameBufferInit/colorSpace}} [=map/exists=], assign its
        value to |colorSpace|.
    11. Assign |init|'s {{VideoFrameBufferInit/format}} to
        {{VideoFrame/[[format]]}}.
    12. Assign the result of running the [=VideoFrame/Pick Color Space=]
        algorithm, with |colorSpace| and {{VideoFrame/[[format]]}}, to
        {{VideoFrame/[[color space]]}}.
    13. Assign the result of calling [=Copy VideoFrame metadata=]
        with |init|'s {{VideoFrameBufferInit/metadata}} to
        |frame|.{{VideoFrame/[[metadata]]}}.
22. Return |frame|.

### Attributes ###{#videoframe-attributes}
: <dfn attribute for=VideoFrame>format</dfn>
:: Describes the arrangement of bytes in each plane as well as the number and
    order of the planes. Will be `null` whenever the underlying format does not
    map to a {{VideoPixelFormat}} or when {{platform object/[[Detached]]}} is
    `true`.

    The {{VideoFrame/format}} getter steps are to return
    {{VideoFrame/[[format]]}}.

: <dfn attribute for=VideoFrame>codedWidth</dfn>
:: Width of the {{VideoFrame}} in pixels, potentially including non-visible
    padding, and prior to considering potential ratio adjustments.

    The {{VideoFrame/codedWidth}} getter steps are to return
    {{VideoFrame/[[coded width]]}}.

: <dfn attribute for=VideoFrame>codedHeight</dfn>
:: Height of the {{VideoFrame}} in pixels, potentially including non-visible
    padding, and prior to considering potential ratio adjustments.

    The {{VideoFrame/codedHeight}} getter steps are to return
    {{VideoFrame/[[coded height]]}}.

: <dfn attribute for=VideoFrame>codedRect</dfn>
:: A {{DOMRectReadOnly}} with {{DOMRectReadOnly/width}} and
    {{DOMRectReadOnly/height}} matching {{VideoFrame/codedWidth}} and
    {{VideoFrame/codedHeight}} and {{DOMRectReadOnly/x}} and
    {{DOMRectReadOnly/y}} at `(0,0)`. Offered for convenience for use with
    {{VideoFrame/allocationSize()}} and {{VideoFrame/copyTo()}}.

    The {{VideoFrame/codedRect}} getter steps are:
    1. If {{platform object/[[Detached]]}} is `true`, return `null`.
    2. Let |rect| be a new {{DOMRectReadOnly}}, initialized as follows:
        1. Assign `0` to {{DOMRectReadOnly/x}} and {{DOMRectReadOnly/y}}.
        2. Assign {{VideoFrame/[[coded width]]}} and
            {{VideoFrame/[[coded height]]}} to {{DOMRectReadOnly/width}} and
            {{DOMRectReadOnly/height}} respectively.
    3. Return |rect|.

: <dfn attribute for=VideoFrame>visibleRect</dfn>
:: A {{DOMRectReadOnly}} describing the visible rectangle of pixels for this
    {{VideoFrame}}.

    The {{VideoFrame/visibleRect}} getter steps are:
    1. If {{platform object/[[Detached]]}} is `true`, return `null`.
    2. Let |rect| be a new {{DOMRectReadOnly}}, initialized as follows:
        1. Assign {{VideoFrame/[[visible left]]}},
            {{VideoFrame/[[visible top]]}}, {{VideoFrame/[[visible width]]}},
            and {{VideoFrame/[[visible height]]}} to {{DOMRectReadOnly/x}},
            {{DOMRectReadOnly/y}}, {{DOMRectReadOnly/width}}, and
            {{DOMRectReadOnly/height}} respectively.
    3. Return |rect|.

: <dfn attribute for=VideoFrame>rotation</dfn>
:: The rotation to applied to the VideoFrame when rendered, in degrees
    clockwise. Rotation applies before flip.

    The {{VideoFrame/rotation}} getter steps are to return
    {{VideoFrame/[[rotation]]}}.

: <dfn attribute for=VideoFrame>flip</dfn>
:: Whether a horizontal flip is applied to the {{VideoFrame}} when rendered.
    Flip applies after rotation.

    The {{VideoFrame/flip}} getter steps are to return {{VideoFrame/[[flip]]}}.

: <dfn attribute for=VideoFrame>displayWidth</dfn>
:: Width of the VideoFrame when displayed after applying rotation and aspect
    ratio adjustments.

    The {{VideoFrame/displayWidth}} getter steps are to return
    {{VideoFrame/[[display width]]}}.

: <dfn attribute for=VideoFrame>displayHeight</dfn>
:: Height of the VideoFrame when displayed after applying rotation and aspect
    ratio adjustments.

    The {{VideoFrame/displayHeight}} getter steps are to return
    {{VideoFrame/[[display height]]}}.

: <dfn attribute for=VideoFrame>timestamp</dfn>
:: The presentation timestamp, given in microseconds. For decode,
    timestamp is copied from the {{EncodedVideoChunk}} corresponding
    to this {{VideoFrame}}. For encode, timestamp is copied to the
    {{EncodedVideoChunk}}s corresponding to this {{VideoFrame}}.

    The {{VideoFrame/timestamp}} getter steps are to return
    {{VideoFrame/[[timestamp]]}}.

: <dfn attribute for=VideoFrame>duration</dfn>
:: The presentation duration, given in microseconds. The duration is copied
    from the {{EncodedVideoChunk}} corresponding to this VideoFrame.

    The {{VideoFrame/duration}} getter steps are to return
    {{VideoFrame/[[duration]]}}.

: <dfn attribute for=VideoFrame>colorSpace</dfn>
:: The {{VideoColorSpace}} associated with this frame.

    The {{VideoFrame/colorSpace}} getter steps are to return
    {{VideoFrame/[[color space]]}}.

### Internal Structures ###{#videoframe-internal-structures}
A <dfn>combined buffer layout</dfn> is a [=struct=] that consists of:
  * A <dfn for="combined buffer layout">allocationSize</dfn> (an {{unsigned
    long}})
  * A <dfn for="combined buffer layout">computedLayouts</dfn> (a [=list=] of
    [=computed plane layout=] structs).

A <dfn>computed plane layout</dfn> is a [=struct=] that consists of:
  * A <dfn for="computed plane layout">destinationOffset</dfn> (an
    {{unsigned long}})
  * A <dfn for="computed plane layout">destinationStride</dfn> (an
    {{unsigned long}})
  * A <dfn for="computed plane layout">sourceTop</dfn> (an {{unsigned long}})
  * A <dfn for="computed plane layout">sourceHeight</dfn> (an {{unsigned long}})
  * A <dfn for="computed plane layout">sourceLeftBytes</dfn> (an
    {{unsigned long}})
  * A <dfn for="computed plane layout">sourceWidthBytes</dfn> (an
    {{unsigned long}})

### Methods ###{#videoframe-methods}
: <dfn method for=VideoFrame>allocationSize(|options|)</dfn>
:: Returns the minimum byte length for a valid destination {{BufferSource}}
    to be used with {{VideoFrame/copyTo()}} with the given options.

    When invoked, run these steps:
    1. If {{platform object/[[Detached]]}} is `true`, throw an
        {{InvalidStateError}} {{DOMException}}.
    2. If {{VideoFrame/[[format]]}} is `null`, throw a {{NotSupportedError}}
        {{DOMException}}.
    3. Let |combinedLayout| be the result of running the [=Parse
        VideoFrameCopyToOptions=] algorithm with |options|.
    4. If |combinedLayout| is an exception, throw |combinedLayout|.
    5. Return |combinedLayout|'s [=combined buffer layout/allocationSize=].

: <dfn method for=VideoFrame>copyTo(|destination|, |options|)</dfn>
:: Asynchronously copies the planes of this frame into |destination| according
    to |options|. The format of the data is |options|.{{VideoFrameCopyToOptions/format}},
    if it [=map/exists=] or [=this=] {{VideoFrame}}'s {{VideoFrame/format}} otherwise.

    NOTE: Promises that are returned by several calls to
        {{VideoFrame/copyTo()}} are not guaranteed to resolve in the order they
        were returned.

    When invoked, run these steps:
    1. If {{platform object/[[Detached]]}} is `true`, return a promise rejected
        with a {{InvalidStateError}} {{DOMException}}.
    2. If {{VideoFrame/[[format]]}} is `null`, return a promise rejected with a
        {{NotSupportedError}} {{DOMException}}.
    3. Let |combinedLayout| be the result of running the [=Parse
        VideoFrameCopyToOptions=] algorithm with |options|.
    4. If |combinedLayout| is an exception, return a promise rejected with
        |combinedLayout|.
    5. If `destination.byteLength` is less than |combinedLayout|'s [=combined
        buffer layout/allocationSize=], return a promise rejected with a
        {{TypeError}}.
    6. If |options|.{{VideoFrameCopyToOptions/format}} is equal to one of
        {{RGBA}}, {{RGBX}}, {{BGRA}}, {{BGRX}} then:
          1. Let |newOptions| be the result of running the <a>Clone Configuration</a>
               algorithm with |options|.
          2. Assign `undefined` to |newOptions|.{{VideoFrameCopyToOptions/format}}.
          3. Let |rgbFrame| be the result of running the [=VideoFrame/Convert to RGB frame=]
               algorithm with [=this=], |options|.{{VideoFrameCopyToOptions/format}},
               and |options|.{{VideoFrameCopyToOptions/colorSpace}}.
          4. Return the result of calling {{VideoFrame/copyTo()}} on |rgbFrame| with
               |destination| and |newOptions|.
    7. Let |p| be a new {{Promise}}.
    8. Let |copyStepsQueue| be the result of starting a new [=parallel queue=].
    9. Let |planeLayouts| be a new [=list=].
    10. Enqueue the following steps to |copyStepsQueue|:
        1. Let resource be the [=media resource=] referenced by
            {{VideoFrame/[[resource reference]]}}.
        2. Let |numPlanes| be the number of planes as defined by
            {{VideoFrame/[[format]]}}.
        3. Let |planeIndex| be `0`.
        4. While |planeIndex| is less than |combinedLayout|'s |numPlanes|:
            1. Let |sourceStride| be the stride of the plane in |resource| as
                identified by |planeIndex|.
            2. Let |computedLayout| be the [=computed plane layout=] in
                |combinedLayout|'s [=combined buffer layout/computedLayouts=] at
                the position of |planeIndex|
            3. Let |sourceOffset| be the product of multiplying
                |computedLayout|'s [=computed plane layout/sourceTop=] by
                |sourceStride|
            4. Add |computedLayout|'s [=computed plane layout/sourceLeftBytes=]
                to |sourceOffset|.
            5. Let |destinationOffset| be |computedLayout|'s
                [=computed plane layout/destinationOffset=].
            6. Let |rowBytes| be |computedLayout|'s
                [=computed plane layout/sourceWidthBytes=].
            7. Let |layout| be a new {{PlaneLayout}}, with
                {{PlaneLayout/offset}} set to |destinationOffset| and
                {{PlaneLayout/stride}} set to |rowBytes|.
            8. Let |row| be `0`.
            9. While |row| is less than |computedLayout|'s
                [=computed plane layout/sourceHeight=]:
                1. Copy |rowBytes| bytes from |resource| starting at
                    |sourceOffset| to |destination| starting at
                    |destinationOffset|.
                2. Increment |sourceOffset| by |sourceStride|.
                3. Increment |destinationOffset| by |computedLayout|'s
                    [=computed plane layout/destinationStride=].
                4. Increment |row| by `1`.
            10. Increment |planeIndex| by `1`.
            11. Append |layout| to |planeLayouts|.
        5. [=Queue a task=] to resolve |p| with |planeLayouts|.
    11. Return |p|.

: <dfn method for=VideoFrame>clone()</dfn>
:: Creates a new {{VideoFrame}} with a reference to the same
    [=media resource=].

    When invoked, run these steps:
    1. If the value of |frame|’s {{platform object/[[Detached]]}} internal slot is
        `true`, throw an {{InvalidStateError}} {{DOMException}}.
    2. Return the result of running the [=Clone VideoFrame=] algorithm with
        [=this=].

: <dfn method for=VideoFrame>close()</dfn>
:: Clears all state and releases the reference to the [=media resource=].
    Close is final.

    When invoked, run the [=Close VideoFrame=] algorithm with [=this=].

: <dfn method for=VideoFrame>metadata()</dfn>
:: Gets the {{VideoFrameMetadata}} associated with this frame.

    When invoked, run these steps:
    1. If {{platform object/[[Detached]]}} is <code>true</code>,
        throw an {{InvalidStateError}} {{DOMException}}.
    2. Return the result of calling [=Copy VideoFrame metadata=]
        with {{VideoFrame/[[metadata]]}}.

### Algorithms ###{#videoframe-algorithms}
: <dfn>Create a VideoFrame</dfn> (with |output|, |timestamp|, |duration|, |displayAspectWidth|, |displayAspectHeight|, |colorSpace|, |rotation|, and |flip|)
:: 1. Let |frame| be a new {{VideoFrame}}, constructed as follows:
        1. Assign `false` to {{platform object/[[Detached]]}}.
        2. Let |resource| be the [=media resource=] described by |output|.
        3. Let |resourceReference| be a reference to |resource|.
        4. Assign |resourceReference| to {{VideoFrame/[[resource reference]]}}.
        5. If |output| uses a recognized {{VideoPixelFormat}}, assign that format to
            {{VideoFrame/[[format]]}}. Otherwise, assign `null` to
            {{VideoFrame/[[format]]}}.
        6. Let |codedWidth| and |codedHeight| be the coded width and height of the
            |output| in pixels.
        8. Let |visibleLeft|, |visibleTop|, |visibleWidth|, and |visibleHeight| be
            the left, top, width and height for the visible rectangle of |output|.
        7. Let |displayWidth| and |displayHeight| be the display size of
            |output| in pixels.
        8. If |displayAspectWidth| and |displayAspectHeight| are provided,
            increase |displayWidth| or |displayHeight| until the ratio of
            |displayWidth| to |displayHeight| matches the ratio of
            |displayAspectWidth| to |displayAspectHeight|.
        9. Assign |codedWidth|, |codedHeight|, |visibleLeft|, |visibleTop|,
            |visibleWidth|, |visibleHeight|, |displayWidth|, and
            |displayHeight| to {{VideoFrame/[[coded width]]}},
            {{VideoFrame/[[coded height]]}}, {{VideoFrame/[[visible left]]}},
            {{VideoFrame/[[visible top]]}}, {{VideoFrame/[[visible width]]}},
            and {{VideoFrame/[[visible height]]}} respectively.
        10. Assign |duration| and |timestamp| to {{VideoFrame/[[duration]]}} and
            {{VideoFrame/[[timestamp]]}} respectively.
        11. Assign {{VideoFrame/[[color space]]}} with the result of running the
            [=VideoFrame/Pick Color Space=] algorithm, with |colorSpace| and
            {{VideoFrame/[[format]]}}.
        12. Assign {{VideoFrame/rotation}} and {{VideoFrame/flip}} to |rotation|
            and |flip| respectively.
    2. Return |frame|.


: <dfn for=VideoFrame>Pick Color Space</dfn> (with |overrideColorSpace| and |format|)
:: 1. If |overrideColorSpace| is provided, return a new {{VideoColorSpace}}
        constructed with |overrideColorSpace|.

        User Agents <em class="rfc2119">MAY</em> replace `null` members of the
        provided |overrideColorSpace| with guessed values as determined by implementer
        defined heuristics.

    2. Otherwise, if {{VideoFrame/[[format]]}} is an [=RGB format=] return a new
        instance of the [=sRGB Color Space=]
    3. Otherwise, return a new instance of the [=REC709 Color Space=].

: <dfn>Validate VideoFrameInit</dfn> (with |format|, |codedWidth|, and
    |codedHeight|):
:: 1. If {{VideoFrameInit/visibleRect}} [=map/exists=]:
        1. Let |validAlignment| be the result of running the
            [=VideoFrame/Verify Rect Offset Alignment=] with |format| and
            |visibleRect|.
        2. If |validAlignment| is `false`, return `false`.
        3. If any attribute of {{VideoFrameInit/visibleRect}} is negative or
            not finite, return `false`.
        4. If {{VideoFrameInit/visibleRect}}.{{DOMRectInit/width}} == `0` or
            {{VideoFrameInit/visibleRect}}.{{DOMRectInit/height}} == `0` return
            `false`.
        5. If {{VideoFrameInit/visibleRect}}.{{DOMRectInit/y}} +
            {{VideoFrameInit/visibleRect}}.{{DOMRectInit/height}} >
            |codedHeight|, return `false`.
        6. If {{VideoFrameInit/visibleRect}}.{{DOMRectInit/x}} +
            {{VideoFrameInit/visibleRect}}.{{DOMRectInit/width}} >
            |codedWidth|, return `false`.
    2. If |codedWidth| = 0 or |codedHeight| = 0,return `false`.
    3. If only one of {{VideoFrameInit/displayWidth}} or
        {{VideoFrameInit/displayHeight}} [=map/exists=], return `false`.
    4. If {{VideoFrameInit/displayWidth}} == `0` or
        {{VideoFrameInit/displayHeight}} == `0`, return `false`.
    5. Return `true`.

: To check if a {{VideoFrameBufferInit}} is a
    <dfn>valid VideoFrameBufferInit</dfn>, run these steps:
:: 1. If {{VideoFrameBufferInit/codedWidth}} = 0 or
        {{VideoFrameBufferInit/codedHeight}} = 0,return `false`.
    2. If any attribute of {{VideoFrameBufferInit/visibleRect}} is negative or
        not finite, return `false`.
    3. If {{VideoFrameBufferInit/visibleRect}}.{{DOMRectInit/y}} +
        {{VideoFrameBufferInit/visibleRect}}.{{DOMRectInit/height}} >
        {{VideoFrameBufferInit/codedHeight}}, return `false`.
    4. If {{VideoFrameBufferInit/visibleRect}}.{{DOMRectInit/x}} +
        {{VideoFrameBufferInit/visibleRect}}.{{DOMRectInit/width}} >
        {{VideoFrameBufferInit/codedWidth}}, return `false`.
    5. If only one of {{VideoFrameBufferInit/displayWidth}} or
        {{VideoFrameBufferInit/displayHeight}} [=map/exists=], return `false`.
    6. If {{VideoFrameBufferInit/displayWidth}} = 0 or
        {{VideoFrameBufferInit/displayHeight}} = 0, return `false`.
    7. Return `true`.

: <dfn for=VideoFrame>Initialize Frame From Other Frame</dfn> (with |init|,
    |frame|, and |otherFrame|)
:: 1. Let |format| be |otherFrame|.{{VideoFrame/format}}.
    2. If |init|.{{VideoFrameInit/alpha}} is {{AlphaOption/discard}},
        assign |otherFrame|.{{VideoFrame/format}}'s [=equivalent opaque format=]
        |format|.
    3. Let |validInit| be the result of running the [=Validate VideoFrameInit=]
        algorithm with |format| and |otherFrame|'s
        {{VideoFrame/[[coded width]]}} and {{VideoFrame/[[coded height]]}}.
    4. If |validInit| is `false`, throw a  {{TypeError}}.
    5. Let |resource| be the [=media resource=] referenced by |otherFrame|'s
        {{VideoFrame/[[resource reference]]}}.
    6. Assign a new reference for |resource| to |frame|'s
        {{VideoFrame/[[resource reference]]}}.
    7. Assign the following attributes from |otherFrame| to |frame|:
        {{VideoFrame/codedWidth}}, {{VideoFrame/codedHeight}},
        {{VideoFrame/colorSpace}}.
    8. Let |defaultVisibleRect| be the result of performing the getter steps
        for {{VideoFrame/visibleRect}} on |otherFrame|.
    9. Let |baseRotation| and |baseFlip| be |otherFrame|'s
        {{VideoFrame/[[rotation]]}} and {{VideoFrame/[[flip]]}}, respectively.
    10. Let |defaultDisplayWidth| and |defaultDisplayHeight| be |otherFrame|'s
        {{VideoFrame/[[display width]]}} and {{VideoFrame/[[display height]]}},
        respectively.
    11. Run the [=VideoFrame/Initialize Visible Rect, Orientation, and Display Size=]
        algorithm with |init|, |frame|, |defaultVisibleRect|, |baseRotation|,
        |baseFlip|, |defaultDisplayWidth|, and |defaultDisplayHeight|.
    12. If {{VideoFrameInit/duration}} [=map/exists=] in |init|, assign it to
        |frame|'s {{VideoFrame/[[duration]]}}. Otherwise, assign
        |otherFrame|.{{VideoFrame/duration}} to
        |frame|'s {{VideoFrame/[[duration]]}}.
    13. If {{VideoFrameInit/timestamp}} [=map/exists=] in |init|, assign it to
        |frame|'s {{VideoFrame/[[timestamp]]}}. Otherwise, assign
        |otherFrame|'s {{VideoFrame/timestamp}} to
        |frame|'s {{VideoFrame/[[timestamp]]}}.
    14. Assign |format| to |frame|.{{VideoFrame/[[format]]}}.
    15. Assign the result of calling [=Copy VideoFrame metadata=]
        with |init|'s {{VideoFrameInit/metadata}} to |frame|.{{VideoFrame/[[metadata]]}}.

: <dfn for=VideoFrame>Initialize Frame With Resource</dfn> (with
    |init|,  |frame|, |resource|, |codedWidth|, |codedHeight|, |baseRotation|,
    |baseFlip|, |defaultDisplayWidth|, and |defaultDisplayHeight|)
:: 1. Let |format| be `null`.
    2. If |resource| uses a recognized {{VideoPixelFormat}}, assign the
        {{VideoPixelFormat}} of |resource| to |format|.
    3. Let |validInit| be the result of running the [=Validate VideoFrameInit=]
        algorithm with |format|, |width| and |height|.
    4. If |validInit| is `false`, throw a  {{TypeError}}.
    5. Assign a new reference for |resource| to |frame|'s
        {{VideoFrame/[[resource reference]]}}.
    6. If |init|.{{VideoFrameInit/alpha}} is {{AlphaOption/discard}}, assign
        |format|'s [=equivalent opaque format=] to |format|.
    7. Assign |format| to {{VideoFrame/[[format]]}}
    8. Assign |codedWidth| and |codedHeight| to |frame|'s
        {{VideoFrame/[[coded width]]}} and {{VideoFrame/[[coded height]]}}
        respectively.
    9. Let |defaultVisibleRect| be a new {{DOMRect}} constructed with
        «[ "x:" → `0`, "y" → `0`, "width" → |codedWidth|, "height" → |codedHeight| ]»
    10. Run the [=VideoFrame/Initialize Visible Rect, Orientation, and Display Size=]
        algorithm with |init|, |frame|, |defaultVisibleRect|,
        |defaultDisplayWidth|, and |defaultDisplayHeight|.
    11. Assign `init`.{{VideoFrameInit/duration}} to
        |frame|'s {{VideoFrame/[[duration]]}}.
    12. Assign `init`.{{VideoFrameInit/timestamp}} to
        |frame|'s {{VideoFrame/[[timestamp]]}}.
    13. If |resource| has a known {{VideoColorSpace}}, assign its value to
        {{VideoFrame/[[color space]]}}.
    14. Otherwise, assign a new {{VideoColorSpace}}, constructed with an empty
        {{VideoColorSpaceInit}}, to {{VideoFrame/[[color space]]}}.

: <dfn for=VideoFrame>Initialize Visible Rect, Orientation, and Display Size</dfn>
    (with |init|, |frame|, |defaultVisibleRect|, |baseRotation|, |baseFlip|,
    |defaultDisplayWidth| and |defaultDisplayHeight|)
:: 1. Let |visibleRect| be |defaultVisibleRect|.
    2. If |init|.{{VideoFrameInit/visibleRect}} [=map/exists=]:
        1. If any attribute of |init|.{{VideoFrameInit/visibleRect}} is negative or
            not finite, throw a {{TypeError}}.
        2. Assign |init|.{{VideoFrameInit/visibleRect}} to |visibleRect|.
    3. Assign |visibleRect|'s {{DOMRect/x}}, {{DOMRect/y}}, {{DOMRect/width}},
        and {{DOMRect/height}}, to |frame|'s {{VideoFrame/[[visible left]]}},
        {{VideoFrame/[[visible top]]}}, {{VideoFrame/[[visible width]]}}, and
        {{VideoFrame/[[visible height]]}} respectively.
    4. Let |rotation| be the result of running the [=VideoFrame/Parse Rotation=]
        algorithm, with |init|.{{VideoFrameInit/rotation}}.
    5. Assign the result of running the [=VideoFrame/Add Rotations=] algorithm,
        with |baseRotation|, |baseFlip|, and |rotation|, to |frame|'s
        {{VideoFrame/[[rotation]]}}.
    6. If |baseFlip| is equal to |init|.{{VideoFrameInit/flip}}, assign `false`
        to |frame|'s {{VideoFrame/[[flip]]}}. Otherwise, assign `true` to
        |frame|'s {{VideoFrame/[[flip]]}}.
    7. If {{VideoFrameInit/displayWidth}} and {{VideoFrameInit/displayHeight}}
        [=map/exist=] in |init|, assign them to {{VideoFrame/[[display width]]}}
        and {{VideoFrame/[[display height]]}} respectively.
    8. Otherwise:
        1. If |baseRotation| is equal to `0` or `180`:
            1. Let |widthScale| be the result of dividing |defaultDisplayWidth|
                by |defaultVisibleRect|.{{DOMRect/width}}.
            2. Let |heightScale| be the result of dividing
                |defaultDisplayHeight| by
                |defaultVisibleRect|.{{DOMRect/height}}.
        2. Otherwise:
            1. Let |widthScale| be the result of dividing |defaultDisplayHeight|
                by |defaultVisibleRect|.{{DOMRect/width}}.
            2. Let |heightScale| be the result of dividing |defaultDisplayWidth|
                by |defaultVisibleRect|.{{DOMRect/height}}.
        3. Let |displayWidth| be
            `|frame|'s {{VideoFrame/[[visible width]]}} * |widthScale|`, rounded
            to the nearest integer.
        4. Let |displayHeight| be
            `|frame|'s {{VideoFrame/[[visible height]]}} * |heightScale|`,
            rounded to the nearest integer.
        5. If |rotation| is equal to `0` or `180`:
            1. Assign |displayWidth| to |frame|'s
                {{VideoFrame/[[display width]]}}.
            2. Assign |displayHeight| to |frame|'s
                {{VideoFrame/[[display height]]}}.
        6. Otherwise:
            1. Assign |displayHeight| to |frame|'s
                {{VideoFrame/[[display width]]}}.
            2. Assign |displayWidth| to |frame|'s
                {{VideoFrame/[[display height]]}}.

: <dfn export>Clone VideoFrame</dfn> (with |frame|)
:: 1. Let |clone| be a new {{VideoFrame}} initialized as follows:
        1. Let |resource| be the [=media resource=] referenced by |frame|’s
            {{VideoFrame/[[resource reference]]}}.
        2. Let |newReference| be a new reference to |resource|.
        3. Assign |newReference| to |clone|'s
            {{VideoFrame/[[resource reference]]}}.
        4. Assign all remaining internal slots of |frame| (excluding
            {{VideoFrame/[[resource reference]]}}) to those of the same name
            in |clone|.
    2. Return |clone|.

: <dfn export>Close VideoFrame</dfn> (with |frame|)
:: 1. Assign `null` to |frame|'s {{VideoFrame/[[resource reference]]}}.
    2. Assign `true` to |frame|'s {{platform object/[[Detached]]}}.
    3. Assign `null` to |frame|'s {{VideoFrame/format}}.
    4. Assign `0` to |frame|'s {{VideoFrame/[[coded width]]}},
        {{VideoFrame/[[coded height]]}}, {{VideoFrame/[[visible left]]}},
        {{VideoFrame/[[visible top]]}}, {{VideoFrame/[[visible width]]}},
        {{VideoFrame/[[visible height]]}}, {{VideoFrame/[[rotation]]}},
        {{VideoFrame/[[display width]]}}, and {{VideoFrame/[[display height]]}}.
    5. Assign `false` to |frame|'s {{VideoFrame/[[flip]]}}.
    6. Assign a new {{VideoFrameMetadata}} to |frame|.{{VideoFrame/[[metadata]]}}.

: <dfn for=VideoFrame>Parse Rotation</dfn> (with |rotation|)
::  1. Let |alignedRotation| be the nearest multiple of `90` to |rotation|,
        rounding ties towards positive infinity.
    2. Let |fullTurns| be the greatest multiple of `360` less than or equal to
        |alignedRotation|.
    3. Return `|alignedRotation| - |fullTurns|`.

: <dfn for=VideoFrame>Add Rotations</dfn> (with |baseRotation|, |baseFlip|,
    and |rotation|)
::  1. If |baseFlip| is `false`, let |combinedRotation| be
        `|baseRotation| + |rotation|`. Otherwise, let |combinedRotation| be
        `|baseRotation| - |rotation|`.
    2. Let |fullTurns| be the greatest multiple of `360` less than or equal to
        |combinedRotation|.
    3. Return `|combinedRotation| - |fullTurns|`.

: <dfn for=VideoFrame>Parse VideoFrameCopyToOptions</dfn> (with |options|)
::  1. Let |defaultRect| be the result of performing the getter steps for
        {{VideoFrame/visibleRect}}.
    2. Let |overrideRect| be `undefined`.
    3. If |options|.{{VideoFrameCopyToOptions/rect}} [=map/exists=], assign the
        value of |options|.{{VideoFrameCopyToOptions/rect}} to |overrideRect|.
    4. Let |parsedRect| be the result of running the [=VideoFrame/Parse Visible
        Rect=] algorithm with |defaultRect|, |overrideRect|,
        {{VideoFrame/[[coded width]]}}, {{VideoFrame/[[coded height]]}}, and
        {{VideoFrame/[[format]]}}.
    5. If |parsedRect| is an exception, return |parsedRect|.
    6. Let |optLayout| be `undefined`.
    7. If |options|.{{VideoFrameCopyToOptions/layout}} [=map/exists=], assign
        its value to |optLayout|.
    8. Let |format| be `undefined`.
    9. If |options|.{{VideoFrameCopyToOptions/format}} does not [=map/exist=],
        assign {{VideoFrame/[[format]]}} to |format|.
    10. Otherwise, if |options|.{{VideoFrameCopyToOptions/format}} is equal to
        one of {{RGBA}}, {{RGBX}}, {{BGRA}}, {{BGRX}}, then assign
        |options|.{{VideoFrameCopyToOptions/format}} to |format|,
        otherwise return {{NotSupportedError}}.
    11. Let |combinedLayout| be the result of running the [=VideoFrame/Compute
        Layout and Allocation Size=] algorithm with |parsedRect|, |format|,
        and |optLayout|.
    12. Return |combinedLayout|.

: <dfn for=VideoFrame>Verify Rect Offset Alignment</dfn> (with |format| and
    |rect|)
:: 1. If |format| is `null`, return `true`.
    2. Let |planeIndex| be `0`.
    3. Let |numPlanes| be the number of planes as defined by |format|.
    4. While |planeIndex| is less than |numPlanes|:
        1. Let |plane| be the Plane identified by |planeIndex| as defined by
            |format|.
        2. Let |sampleWidth| be the horizontal [=sub-sampling factor=] of each
            subsample for |plane|.
        3. Let |sampleHeight| be the vertical [=sub-sampling factor=] of each
            subsample for |plane|.
        4. If |rect|.{{DOMRectReadOnly/x}} is not a multiple of |sampleWidth|,
            return `false`.
        5. If |rect|.{{DOMRectReadOnly/y}} is not a multiple of |sampleHeight|,
            return `false`.
        8. Increment |planeIndex| by `1`.
    5. Return `true`.

: <dfn for=VideoFrame>Parse Visible Rect</dfn> (with |defaultRect|,
    |overrideRect|, |codedWidth|, |codedHeight|, and |format|)
:: 1. Let |sourceRect| be |defaultRect|
    2. If |overrideRect| is not `undefined`:
        1. If any attribute of |overrideRect| is negative or not finite, return a {{TypeError}}.
        2. If either of |overrideRect|.{{DOMRectInit/width}} or
            {{DOMRectInit/height}} is `0`, return a {{TypeError}}.
        3. If the sum of |overrideRect|.{{DOMRectInit/x}} and
            |overrideRect|.{{DOMRectInit/width}} is greater than
            |codedWidth|, return a {{TypeError}}.
        4. If the sum of |overrideRect|.{{DOMRectInit/y}} and
            |overrideRect|.{{DOMRectInit/height}} is greater than
            |codedHeight|, return a {{TypeError}}.
        5. Assign |overrideRect| to |sourceRect|.
    3. Let |validAlignment| be the result of running the
        [=VideoFrame/Verify Rect Offset Alignment=] algorithm with |format| and
        |sourceRect|.
    4. If |validAlignment| is `false`, throw a {{TypeError}}.
    5. Return |sourceRect|.

: <dfn for=VideoFrame>Compute Layout and Allocation Size</dfn> (with
    |parsedRect|, |format|, and |layout|)
:: 1. Let |numPlanes| be the number of planes as defined by |format|.
    2. If |layout| is not `undefined` and its length does not equal |numPlanes|,
         throw a {{TypeError}}.
    3. Let |minAllocationSize| be `0`.
    4. Let |computedLayouts| be a new [=list=].
    5. Let |endOffsets| be a new [=list=].
    6. Let |planeIndex| be `0`.
    7. While |planeIndex| < |numPlanes|:
        1. Let |plane| be the Plane identified by |planeIndex| as defined by
            |format|.
        2. Let |sampleBytes| be the number of bytes per sample for |plane|.
        3. Let |sampleWidth| be the horizontal [=sub-sampling factor=] of each
            subsample for |plane|.
        4. Let |sampleHeight| be the vertical [=sub-sampling factor=] of each
            subsample for |plane|.
        5. Let |computedLayout| be a new [=computed plane layout=].
        6. Set |computedLayout|'s [=computed plane layout/sourceTop=] to the
            result of the division of truncated |parsedRect|.{{DOMRectInit/y}}
            by |sampleHeight|, rounded up to the nearest integer.
        7. Set |computedLayout|'s [=computed plane layout/sourceHeight=] to the
            result of the division of truncated
            |parsedRect|.{{DOMRectInit/height}} by |sampleHeight|, rounded up
            to the nearest integer.
        8. Set |computedLayout|'s [=computed plane layout/sourceLeftBytes=] to
            the result of the integer division of
            truncated |parsedRect|.{{DOMRectInit/x}} by |sampleWidth|,
            multiplied by |sampleBytes|.
        9. Set |computedLayout|'s [=computed plane layout/sourceWidthBytes=] to
            the result of the integer division of
            truncated |parsedRect|.{{DOMRectInit/width}} by |sampleWidth|,
            multiplied by |sampleBytes|.
        10. If |layout| is not `undefined`:
            1. Let |planeLayout| be the {{PlaneLayout}} in |layout| at position
                |planeIndex|.
            2. If |planeLayout|.{{PlaneLayout/stride}} is less than
                |computedLayout|'s [=computed plane layout/sourceWidthBytes=],
                return a {{TypeError}}.
            3. Assign |planeLayout|.{{PlaneLayout/offset}} to |computedLayout|'s
                [=computed plane layout/destinationOffset=].
            4. Assign |planeLayout|.{{PlaneLayout/stride}} to |computedLayout|'s
                [=computed plane layout/destinationStride=].
        11. Otherwise:

            NOTE: If an explicit layout was not provided, the following steps
                default to tight packing.

            1. Assign |minAllocationSize| to |computedLayout|'s
                [=computed plane layout/destinationOffset=].
            2. Assign |computedLayout|'s
                [=computed plane layout/sourceWidthBytes=] to
                |computedLayout|'s [=computed plane layout/destinationStride=].
        12. Let |planeSize| be the product of multiplying |computedLayout|'s
                [=computed plane layout/destinationStride=] and
                [=computed plane layout/sourceHeight=].
        13. Let |planeEnd| be the sum of |planeSize| and |computedLayout|'s
                [=computed plane layout/destinationOffset=].
        14. If |planeSize| or |planeEnd| is greater than maximum range of
                {{unsigned long}}, return a {{TypeError}}.
        15. Append |planeEnd| to |endOffsets|.
        16. Assign the maximum of |minAllocationSize| and |planeEnd| to
                |minAllocationSize|.

                NOTE: The above step uses a maximum to allow for the
                    possibility that user specified plane offsets reorder
                    planes.

        17. Let |earlierPlaneIndex| be `0`.
        18. While |earlierPlaneIndex| is less than |planeIndex|.
            1. Let |earlierLayout| be `computedLayouts[earlierPlaneIndex]`.
            2. If `endOffsets[planeIndex]` is less than or equal to
                |earlierLayout|'s [=computed plane layout/destinationOffset=] or
                if `endOffsets[earlierPlaneIndex]` is less than or equal to
                |computedLayout|'s [=computed plane layout/destinationOffset=],
                continue.

                NOTE: If plane A ends before plane B starts, they do not
                    overlap.

            3. Otherwise, return a {{TypeError}}.
            4. Increment |earlierPlaneIndex| by `1`.
        19. Append |computedLayout| to |computedLayouts|.
        20. Increment |planeIndex| by `1`.
    8. Let |combinedLayout| be a new [=combined buffer layout=], initialized as
        follows:
        1. Assign |computedLayouts| to
            [=combined buffer layout/computedLayouts=].
        2. Assign |minAllocationSize| to
            [=combined buffer layout/allocationSize=].
    9. Return |combinedLayout|.

: <dfn>Convert PredefinedColorSpace to VideoColorSpace</dfn> (with |colorSpace|)
 :: 1. Assert: |colorSpace| is equal to one of {{srgb}} or {{display-p3}}.
    2. If |colorSpace| is equal to {{srgb}} return a new instance of the
        [=sRGB Color Space=]
    3. If |colorSpace| is equal to {{display-p3}} return a new instance of the
        [=Display P3 Color Space=]

: <dfn for=VideoFrame>Convert to RGB frame</dfn> (with |frame|, |format| and |colorSpace|)
 :: 1. This algorithm <em class="rfc2119">MUST</em> be called only if |format|
        is equal to one of {{RGBA}}, {{RGBX}}, {{BGRA}}, {{BGRX}}.
    2. Let |convertedFrame| be a new {{VideoFrame}}, constructed as follows:
        1. Assign `false` to {{platform object/[[Detached]]}}.
        2. Assign |format| to {{VideoFrame/[[format]]}}.
        3. Let |width| be |frame|'s {{VideoFrame/[[visible width]]}}.
        4. Let |height| be |frame|'s {{VideoFrame/[[visible height]]}}.
        5. Assign |width|, |height|, 0, 0, |width|, |height|, |width|, and
            |height| to {{VideoFrame/[[coded width]]}},
            {{VideoFrame/[[coded height]]}}, {{VideoFrame/[[visible left]]}},
            {{VideoFrame/[[visible top]]}}, {{VideoFrame/[[visible width]]}},
            and {{VideoFrame/[[visible height]]}} respectively.
        6. Assign |frame|'s {{VideoFrame/[[duration]]}} and |frame|'s
            {{VideoFrame/[[timestamp]]}} to {{VideoFrame/[[duration]]}} and
            {{VideoFrame/[[timestamp]]}} respectively.
        7. Assign the result of running the <a>Convert
            PredefinedColorSpace to VideoColorSpace</a> algorithm with
            |colorSpace| to {{VideoFrame/[[color space]]}}.
        8. Let |resource| be a new [=media resource=] containing the result of
            conversion of [=media resource=] referenced by |frame|'s
            {{VideoFrame/[[resource reference]]}} into a color space and pixel
            format specified by {{VideoFrame/[[color space]]}} and
            {{VideoFrame/[[format]]}} respectively.
        9. Assign the reference to |resource| to {{VideoFrame/[[resource reference]]}}
    3. Return |convertedFrame|.

: <dfn for=VideoFrame>Copy VideoFrame metadata</dfn> (with |metadata|)
:: 1. Let |metadataCopySerialized| be [$StructuredSerialize$](|metadata|).
    2. Let |metadataCopy| be [$StructuredDeserialize$](|metadataCopySerialized|, [=the current Realm=]).
    3. Return |metadataCopy|.

The goal of this algorithm is to ensure that metadata owned by a {{VideoFrame}} is immutable.

### Transfer and Serialization ###{#videoframe-transfer-serialization}

: The {{VideoFrame}} [=transfer steps=] (with |value| and |dataHolder|) are:
:: 1. If |value|'s {{platform object/[[Detached]]}} is `true`, throw a
        {{DataCloneError}} {{DOMException}}.
    2. For all {{VideoFrame}} internal slots in |value|, assign the value of
        each internal slot to a field in |dataHolder| with the same name as the
        internal slot.
    3. Run the [=Close VideoFrame=] algorithm with |value|.

: The {{VideoFrame}} [=transfer-receiving steps=] (with |dataHolder| and
    |value|) are:
:: 1. For all named fields in |dataHolder|, assign the value of each named
        field to the {{VideoFrame}} internal slot in |value| with the same name
        as the named field.

: The {{VideoFrame}} [=serialization steps=] (with |value|, |serialized|, and
    |forStorage|) are:
:: 1. If |value|'s {{platform object/[[Detached]]}} is `true`, throw a
        {{DataCloneError}} {{DOMException}}.
    2. If |forStorage| is `true`, throw a {{DataCloneError}}.
    3. Let |resource| be the [=media resource=] referenced by
            |value|'s {{VideoFrame/[[resource reference]]}}.
    4. Let |newReference| be a new reference to |resource|.
    5. Assign |newReference| to |serialized.resource reference|.
    6. For all remaining {{VideoFrame}} internal slots (excluding
        {{VideoFrame/[[resource reference]]}}) in |value|, assign the value of
        each internal slot to a field in |serialized| with the same name as the
        internal slot.

: The {{VideoFrame}} [=deserialization steps=] (with |serialized| and |value|)
    are:
:: 1. For all named fields in |serialized|, assign the value of each named
        field to the {{VideoFrame}} internal slot in |value| with the same name
        as the named field.

### Rendering ### {#videoframe-rendering}

When rendered, for example by {{CanvasDrawImage}}
{{CanvasDrawImage/drawImage()}}, a {{VideoFrame}} <em class="rfc2119">MUST</em>
be converted to a color space compatible with the rendering target, unless color
conversion is explicitly disabled.

Color space conversion during {{ImageBitmap}} construction is controlled by
{{ImageBitmapOptions}} {{ImageBitmapOptions/colorSpaceConversion}}. Setting this
value to {{ColorSpaceConversion/"none"}} disables color space conversion.

The rendering of a {{VideoFrame}} is produced from the [=media resource=] by
applying any necessary color space conversion, cropping to the
{{VideoFrame/visibleRect}}, rotating clockwise by {{VideoFrame/rotation}}
degrees, and flipping horizontally if {{VideoFrame/flip}} is `true`.

VideoFrame CopyTo() Options {#videoframe-copyto-options}
------------------------------------------------------------
Options to specify a rectangle of pixels to copy, their format, and the offset
and stride of planes in the destination buffer.

<xmp class='idl'>
dictionary VideoFrameCopyToOptions {
  DOMRectInit rect;
  sequence<PlaneLayout> layout;
  VideoPixelFormat format;
  PredefinedColorSpace colorSpace;
};
</xmp>

<div class='note'>
NOTE: The steps of {{VideoFrame/copyTo()}} or {{VideoFrame/allocationSize()}}
will enforce the following requirements:
    * The coordinates of {{VideoFrameCopyToOptions/rect}} are
        sample-aligned as determined by {{VideoFrame/[[format]]}}.
    * If {{VideoFrameCopyToOptions/layout}} [=map/exists=], a {{PlaneLayout}}
        is provided for all planes.
</div>

: <dfn dict-member for=VideoFrameCopyToOptions>rect</dfn>
:: A {{DOMRectInit}} describing the rectangle of pixels to copy from the
    {{VideoFrame}}. If unspecified, the {{VideoFrame/visibleRect}} will be used.

    NOTE: The coded rectangle can be specified by passing {{VideoFrame}}'s
        {{VideoFrame/codedRect}}.

    NOTE: The default {{VideoFrameCopyToOptions/rect}} does not necessarily meet
        the sample-alignment requirement and can result in
        {{VideoFrame/copyTo()}} or {{VideoFrame/allocationSize()}} rejecting.
: <dfn dict-member for=VideoFrameCopyToOptions>layout</dfn>
:: The {{PlaneLayout}} for each plane in {{VideoFrame}}, affording the option
    to specify an offset and stride for each plane in the destination
    {{BufferSource}}. If unspecified, the planes will be tightly packed. It is
    invalid to specify planes that overlap.
: <dfn dict-member for=VideoFrameCopyToOptions>format</dfn>
:: A {{VideoPixelFormat}} for the pixel data in the destination
    {{BufferSource}}. Potential values are: {{RGBA}}, {{RGBX}}, {{BGRA}},
    {{BGRX}}. If it does not [=map/exist=], the destination
    {{BufferSource}} will be in the same format as {{VideoFrame/format}} .
: <dfn dict-member for=VideoFrameCopyToOptions>colorSpace</dfn>
:: A {{PredefinedColorSpace}} that <em class="rfc2119">MUST</em> be used as
    a target color space for the pixel data in the destination
    {{BufferSource}}, but only if {{VideoFrameCopyToOptions/format}} is one of
    {{RGBA}}, {{RGBX}}, {{BGRA}}, {{BGRX}}, otherwise it is ignored.
    If it does not [=map/exist=], {{srgb}} is used.


DOMRects in VideoFrame {#videoframe-domrect}
--------------------------------------------
The {{VideoFrame}} interface uses {{DOMRect}}s to specify the position and
dimensions for a rectangle of pixels. {{DOMRectInit}} is used with
{{VideoFrame/copyTo()}} and {{VideoFrame/allocationSize()}} to describe the
dimensions of the source rectangle. {{VideoFrame}} defines
{{VideoFrame/codedRect}} and {{VideoFrame/visibleRect}} for convenient copying
of the coded size and visible region respectively.

NOTE: VideoFrame pixels are only addressable by integer numbers. All floating
    point values provided to {{DOMRectInit}} will be truncated.

Plane Layout{#plane-layout}
---------------------------
A {{PlaneLayout}} is a dictionary specifying the offset and stride of a
{{VideoFrame}} plane once copied to a {{BufferSource}}. A sequence of
{{PlaneLayout}}s <em class="rfc2119">MAY</em> be provided to {{VideoFrame}}'s
{{VideoFrame/copyTo()}} to specify how the plane is laid out in the destination
{{BufferSource}}.  Alternatively, callers can inspect {{VideoFrame/copyTo()}}'s
returned sequence of {{PlaneLayout}}s to learn the offset and stride for
planes as decided by the User Agent.

<xmp class='idl'>
dictionary PlaneLayout {
  [EnforceRange] required unsigned long offset;
  [EnforceRange] required unsigned long stride;
};
</xmp>

: <dfn dict-member for=PlaneLayout>offset</dfn>
:: The offset in bytes where the given plane begins within a {{BufferSource}}.

: <dfn dict-member for=PlaneLayout>stride</dfn>
:: The number of bytes, including padding, used by each row of the plane within
    a {{BufferSource}}.

Pixel Format{#pixel-format}
---------------------------
Pixel formats describe the arrangement of bytes in each plane as well as the
number and order of the planes. Each format is described in its own sub-section.

<xmp class='idl'>
enum VideoPixelFormat {
  // 4:2:0 Y, U, V
  "I420",
  "I420P10",
  "I420P12",
  // 4:2:0 Y, U, V, A
  "I420A",
  "I420AP10",
  "I420AP12",
  // 4:2:2 Y, U, V
  "I422",
  "I422P10",
  "I422P12",
  // 4:2:2 Y, U, V, A
  "I422A",
  "I422AP10",
  "I422AP12",
  // 4:4:4 Y, U, V
  "I444",
  "I444P10",
  "I444P12",
  // 4:4:4 Y, U, V, A
  "I444A",
  "I444AP10",
  "I444AP12",
  // 4:2:0 Y, UV
  "NV12",
  // 4:4:4 RGBA
  "RGBA",
  // 4:4:4 RGBX (opaque)
  "RGBX",
  // 4:4:4 BGRA
  "BGRA",
  // 4:4:4 BGRX (opaque)
  "BGRX",
};
</xmp>

<dfn lt="sub-sampling|sub-sampled">Sub-sampling</dfn> is a technique
where a single sample contains information for multiple pixels in the final
image. [=Sub-sampling=] can be horizontal, vertical or both, and has a <dfn
lt="sub-sampling factor|factor">factor</dfn>, that is the number of final pixels
in the image that are derived from a [=sub-sampled=] sample.

<div class=example>
  If a {{VideoFrame}} is in {{I420}} format, then the very first
  component of the second plane (the U plane) corresponds to four pixels, that are
  the pixels in the top-left angle of the image. Consequently, the first
  component of the second row corresponds to the four pixels below those initial
  four top-left pixels. The [=sub-sampling factor=] is 2 in both the horizontal
  and vertical direction.
</div>

If a {{VideoPixelFormat}} has an alpha component, the format's
<dfn>equivalent opaque format</dfn> is the same {{VideoPixelFormat}}, without an
alpha component. If a {{VideoPixelFormat}} does not have an alpha component, it
is its own [=equivalent opaque format=].

Integer values are unsigned unless otherwise specified.

<dl>
  <dt><dfn enum-value for=VideoPixelFormat>I420</dfn></dt>
  <dd>
    This format is composed of three distinct planes, one plane of Luma and two
    planes of Chroma, denoted Y, U and V, and present in this order. It is also
    often refered to as Planar YUV 4:2:0.

    The U and V planes are [=sub-sampled=] horizontally and vertically by a
    [=factor=] of 2 compared to the Y plane.

    Each sample in this format is 8 bits.

    There are {{VideoFrame/codedWidth}} * {{VideoFrame/codedHeight}} samples
    (and therefore bytes) in the Y plane, arranged starting at the top left of
    the image, in {{VideoFrame/codedHeight}} rows of {{VideoFrame/codedWidth}}
    samples.

    The U and V planes have a number of rows equal to the result of the
    division of {{VideoFrame/codedHeight}} by 2, rounded up to the nearest
    integer. Each row has a number of samples equal to the result of the
    division of {{VideoFrame/codedWidth}} by 2, rounded up to the nearest
    integer. Samples are arranged starting at the top left of the image.

    The visible rectangle offset ({{VideoFrame/visibleRect}}.{{DOMRectInit/x}}
    and {{VideoFrame/visibleRect}}.{{DOMRectInit/y}})
    <em class="rfc2119">MUST</em> be even.
  </dd>
  <dt><dfn enum-value for=VideoPixelFormat>I420P10</dfn></dt>
  <dd>
    This format is composed of three distinct planes, one plane of Luma and two
    planes of Chroma, denoted Y, U and V, and present in this order.

    The U and V planes are [=sub-sampled=] horizontally and vertically by a
    [=factor=] of 2 compared to the Y plane.

    Each sample in this format is 10 bits, encoded as a 16-bit integer in
    little-endian byte order.

    There are {{VideoFrame/codedWidth}} * {{VideoFrame/codedHeight}} samples
    in the Y plane, arranged starting at the top left of the image, in
    {{VideoFrame/codedHeight}} rows of {{VideoFrame/codedWidth}} samples.

    The U and V planes have a number of rows equal to the result of the
    division of {{VideoFrame/codedHeight}} by 2, rounded up to the nearest
    integer. Each row has a number of samples equal to the result of the
    division of {{VideoFrame/codedWidth}} by 2, rounded up to the nearest
    integer. Samples are arranged starting at the top left of the image.

    The visible rectangle offset ({{VideoFrame/visibleRect}}.{{DOMRectInit/x}}
    and {{VideoFrame/visibleRect}}.{{DOMRectInit/y}})
    <em class="rfc2119">MUST</em> be even.
  </dd>
  <dt><dfn enum-value for=VideoPixelFormat>I420P12</dfn></dt>
  <dd>
    This format is composed of three distinct planes, one plane of Luma and two
    planes of Chroma, denoted Y, U and V, and present in this order.

    The U and V planes are [=sub-sampled=] horizontally and vertically by a
    [=factor=] of 2 compared to the Y plane.

    Each sample in this format is 12 bits, encoded as a 16-bit integer in
    little-endian byte order.

    There are {{VideoFrame/codedWidth}} * {{VideoFrame/codedHeight}} samples
    in the Y plane, arranged starting at the top left of the image, in
    {{VideoFrame/codedHeight}} rows of {{VideoFrame/codedWidth}} samples.

    The U and V planes have a number of rows equal to the result of the
    division of {{VideoFrame/codedHeight}} by 2, rounded up to the nearest
    integer. Each row has a number of samples equal to the result of the
    division of {{VideoFrame/codedWidth}} by 2, rounded up to the nearest
    integer. Samples are arranged starting at the top left of the image.

    The visible rectangle offset ({{VideoFrame/visibleRect}}.{{DOMRectInit/x}}
    and {{VideoFrame/visibleRect}}.{{DOMRectInit/y}})
    <em class="rfc2119">MUST</em> be even.
  </dd>
  <dt><dfn enum-value for=VideoPixelFormat>I420A</dfn></dt>
  <dd>
    This format is composed of four distinct planes, one plane of Luma, two
    planes of Chroma, denoted Y, U and V, and one plane of Alpha values, all
    present in this order. It is also often refered to as Planar YUV 4:2:0 with
    an alpha channel.

    The U and V planes are [=sub-sampled=] horizontally and vertically by a
    [=factor=] of 2 compared to the Y and Alpha planes.

    Each sample in this format is 8 bits.

    There are {{VideoFrame/codedWidth}} * {{VideoFrame/codedHeight}} samples
    (and therefore bytes) in the Y and Alpha planes, arranged starting at the
    top left of the image, in {{VideoFrame/codedHeight}} rows of
    {{VideoFrame/codedWidth}} samples.

    The U and V planes have a number of rows equal to the result of the
    division of {{VideoFrame/codedHeight}} by 2, rounded up to the nearest
    integer. Each row has a number of samples equal to the result of the
    division of {{VideoFrame/codedWidth}} by 2, rounded up to the nearest
    integer. Samples are arranged starting at the top left of the image.

    The visible rectangle offset ({{VideoFrame/visibleRect}}.{{DOMRectInit/x}}
    and {{VideoFrame/visibleRect}}.{{DOMRectInit/y}})
    <em class="rfc2119">MUST</em> be even.

    {{I420A}}'s [=equivalent opaque format=] is {{I420}}.
  </dd>
  <dt><dfn enum-value for=VideoPixelFormat>I420AP10</dfn></dt>
  <dd>
    This format is composed of four distinct planes, one plane of Luma, two
    planes of Chroma, denoted Y, U and V, and one plane of Alpha values, all
    present in this order.

    The U and V planes are [=sub-sampled=] horizontally and vertically by a
    [=factor=] of 2 compared to the Y and Alpha planes.

    Each sample in this format is 10 bits, encoded as a 16-bit integer in
    little-endian byte order.

    There are {{VideoFrame/codedWidth}} * {{VideoFrame/codedHeight}} samples
    in the Y and Alpha planes, arranged starting at the top left of the image,
    in {{VideoFrame/codedHeight}} rows of {{VideoFrame/codedWidth}} samples.

    The U and V planes have a number of rows equal to the result of the
    division of {{VideoFrame/codedHeight}} by 2, rounded up to the nearest
    integer. Each row has a number of samples equal to the result of the
    division of {{VideoFrame/codedWidth}} by 2, rounded up to the nearest
    integer. Samples are arranged starting at the top left of the image.

    The visible rectangle offset ({{VideoFrame/visibleRect}}.{{DOMRectInit/x}}
    and {{VideoFrame/visibleRect}}.{{DOMRectInit/y}})
    <em class="rfc2119">MUST</em> be even.

    {{I420AP10}}'s [=equivalent opaque format=] is {{I420P10}}.
  </dd>
  <dt><dfn enum-value for=VideoPixelFormat>I420AP12</dfn></dt>
  <dd>
    This format is composed of four distinct planes, one plane of Luma, two
    planes of Chroma, denoted Y, U and V, and one plane of Alpha values, all
    present in this order.

    The U and V planes are [=sub-sampled=] horizontally and vertically by a
    [=factor=] of 2 compared to the Y and Alpha planes.

    Each sample in this format is 12 bits, encoded as a 16-bit integer in
    little-endian byte order.

    There are {{VideoFrame/codedWidth}} * {{VideoFrame/codedHeight}} samples
    in the Y and Alpha planes, arranged starting at the top left of the image,
    in {{VideoFrame/codedHeight}} rows of {{VideoFrame/codedWidth}} samples.

    The U and V planes have a number of rows equal to the result of the
    division of {{VideoFrame/codedHeight}} by 2, rounded up to the nearest
    integer. Each row has a number of samples equal to the result of the
    division of {{VideoFrame/codedWidth}} by 2, rounded up to the nearest
    integer. Samples are arranged starting at the top left of the image.

    The visible rectangle offset ({{VideoFrame/visibleRect}}.{{DOMRectInit/x}}
    and {{VideoFrame/visibleRect}}.{{DOMRectInit/y}})
    <em class="rfc2119">MUST</em> be even.

    {{I420AP12}}'s [=equivalent opaque format=] is {{I420P12}}.
  </dd>
  <dt><dfn enum-value for=VideoPixelFormat>I422</dfn></dt>
  <dd>
    This format is composed of three distinct planes, one plane of Luma and two
    planes of Chroma, denoted Y, U and V, and present in this order. It is also
    often refered to as Planar YUV 4:2:2.

    The U and V planes are [=sub-sampled=] horizontally by a [=factor=] of 2
    compared to the Y plane, and not [=sub-sampled=] vertically.

    Each sample in this format is 8 bits.

    There are {{VideoFrame/codedWidth}} * {{VideoFrame/codedHeight}} samples
    (and therefore bytes) in the Y and plane, arranged starting at the top left
    of the image, in {{VideoFrame/codedHeight}} rows of
    {{VideoFrame/codedWidth}} samples.

    The U and V planes have {{VideoFrame/codedHeight}} rows. Each row has a
    number of samples equal to the result of the division of
    {{VideoFrame/codedWidth}} by 2, rounded up to the nearest integer. Samples
    are arranged starting at the top left of the image.

    The visible rectangle horizontal offset
    ({{VideoFrame/visibleRect}}.{{DOMRectInit/x}})
    <em class="rfc2119">MUST</em> be even.
  </dd>
  <dt><dfn enum-value for=VideoPixelFormat>I422P10</dfn></dt>
  <dd>
    This format is composed of three distinct planes, one plane of Luma and two
    planes of Chroma, denoted Y, U and V, and present in this order.

    The U and V planes are [=sub-sampled=] horizontally by a [=factor=] of 2
    compared to the Y plane, and not [=sub-sampled=] vertically.

    Each sample in this format is 10 bits, encoded as a 16-bit integer in
    little-endian byte order.

    There are {{VideoFrame/codedWidth}} * {{VideoFrame/codedHeight}} samples
    in the Y plane, arranged starting at the top left of the image, in
    {{VideoFrame/codedHeight}} rows of {{VideoFrame/codedWidth}} samples.

    The U and V planes have {{VideoFrame/codedHeight}} rows. Each row has a
    number of samples equal to the result of the division of
    {{VideoFrame/codedWidth}} by 2, rounded up to the nearest integer.
    Samples are arranged starting at the top left of the image.

    The visible rectangle horizontal offset
    ({{VideoFrame/visibleRect}}.{{DOMRectInit/x}})
    <em class="rfc2119">MUST</em> be even.
  </dd>
  <dt><dfn enum-value for=VideoPixelFormat>I422P12</dfn></dt>
  <dd>
    This format is composed of three distinct planes, one plane of Luma and two
    planes of Chroma, denoted Y, U and V, and present in this order.

    The U and V planes are [=sub-sampled=] horizontally by a [=factor=] of 2
    compared to the Y plane, and not [=sub-sampled=] vertically.

    Each sample in this format is 12 bits, encoded as a 16-bit integer in
    little-endian byte order.

    There are {{VideoFrame/codedWidth}} * {{VideoFrame/codedHeight}} samples
    in the Y plane, arranged starting at the top left of the image, in
    {{VideoFrame/codedHeight}} rows of {{VideoFrame/codedWidth}} samples.

    The U and V planes have {{VideoFrame/codedHeight}} rows. Each row has a
    number of samples equal to the result of the division of
    {{VideoFrame/codedWidth}} by 2, rounded up to the nearest integer.
    Samples are arranged starting at the top left of the image.

    The visible rectangle horizontal offset
    ({{VideoFrame/visibleRect}}.{{DOMRectInit/x}})
    <em class="rfc2119">MUST</em> be even.
  </dd>
  <dt><dfn enum-value for=VideoPixelFormat>I422A</dfn></dt>
  <dd>
    This format is composed of four distinct planes, one plane of Luma, two
    planes of Chroma, denoted Y, U and V, and one plane of Alpha values, all
    present in this order. It is also often refered to as Planar YUV 4:2:2 with
    an alpha channel.

    The U and V planes are [=sub-sampled=] horizontally by a [=factor=] of 2
    compared to the Y and Alpha planes, and not [=sub-sampled=] vertically.

    Each sample in this format is 8 bits.

    There are {{VideoFrame/codedWidth}} * {{VideoFrame/codedHeight}} samples
    (and therefore bytes) in the Y and Alpha planes, arranged starting at the
    top left of the image, in {{VideoFrame/codedHeight}} rows of
    {{VideoFrame/codedWidth}} samples.

    The U and V planes have {{VideoFrame/codedHeight}} rows. Each row has a
    number of samples equal to the result of the division of
    {{VideoFrame/codedWidth}} by 2, rounded up to the nearest integer. Samples
    are arranged starting at the top left of the image.

    The visible rectangle horizontal offset
    ({{VideoFrame/visibleRect}}.{{DOMRectInit/x}})
    <em class="rfc2119">MUST</em> be even.

    {{I422A}}'s [=equivalent opaque format=] is {{I422}}.
  </dd>
  <dt><dfn enum-value for=VideoPixelFormat>I422AP10</dfn></dt>
  <dd>
    This format is composed of four distinct planes, one plane of Luma, two
    planes of Chroma, denoted Y, U and V, and one plane of Alpha values, all
    present in this order.

    The U and V planes are [=sub-sampled=] horizontally by a [=factor=] of 2
    compared to the Y and Alpha planes, and not [=sub-sampled=] vertically.

    Each sample in this format is 10 bits, encoded as a 16-bit integer in
    little-endian byte order.

    There are {{VideoFrame/codedWidth}} * {{VideoFrame/codedHeight}} samples
    in the Y and Alpha planes, arranged starting at the top left of the image,
    in {{VideoFrame/codedHeight}} rows of {{VideoFrame/codedWidth}} samples.

    The U and V planes have {{VideoFrame/codedHeight}} rows. Each row has a
    number of samples equal to the result of the division of
    {{VideoFrame/codedWidth}} by 2, rounded up to the nearest integer.
    Samples are arranged starting at the top left of the image.

    The visible rectangle horizontal offset
    ({{VideoFrame/visibleRect}}.{{DOMRectInit/x}})
    <em class="rfc2119">MUST</em> be even.

    {{I422AP10}}'s [=equivalent opaque format=] is {{I420P10}}.
  </dd>
  <dt><dfn enum-value for=VideoPixelFormat>I422AP12</dfn></dt>
  <dd>
    This format is composed of four distinct planes, one plane of Luma, two
    planes of Chroma, denoted Y, U and V, and one plane of Alpha values, all
    present in this order.

    The U and V planes are [=sub-sampled=] horizontally by a [=factor=] of 2
    compared to the Y and Alpha planes, and not [=sub-sampled=] vertically.

    Each sample in this format is 12 bits, encoded as a 16-bit integer in
    little-endian byte order.

    There are {{VideoFrame/codedWidth}} * {{VideoFrame/codedHeight}} samples
    in the Y and Alpha planes, arranged starting at the top left of the image,
    in {{VideoFrame/codedHeight}} rows of {{VideoFrame/codedWidth}} samples.

    The U and V planes have {{VideoFrame/codedHeight}} rows. Each row has a
    number of samples equal to the result of the division of
    {{VideoFrame/codedWidth}} by 2, rounded up to the nearest integer.
    Samples are arranged starting at the top left of the image.

    The visible rectangle horizontal offset
    ({{VideoFrame/visibleRect}}.{{DOMRectInit/x}})
    <em class="rfc2119">MUST</em> be even.

    {{I422AP10}}'s [=equivalent opaque format=] is {{I420P10}}.
  </dd>
  <dt><dfn enum-value for=VideoPixelFormat>I444</dfn></dt>
  <dd>
    This format is composed of three distinct planes, one plane of Luma and two
    planes of Chroma, denoted Y, U and V, and present in this order. It is also
    often refered to as Planar YUV 4:4:4.

    This format does not use [=sub-sampling=].

    Each sample in this format is 8 bits.

    There are {{VideoFrame/codedWidth}} * {{VideoFrame/codedHeight}} samples
    (and therefore bytes) in all three planes, arranged starting at the top left
    of the image, in {{VideoFrame/codedHeight}} rows of
    {{VideoFrame/codedWidth}} samples.
  </dd>
  <dt><dfn enum-value for=VideoPixelFormat>I444P10</dfn></dt>
  <dd>
    This format is composed of three distinct planes, one plane of Luma and two
    planes of Chroma, denoted Y, U and V, and present in this order.

    This format does not use [=sub-sampling=].

    Each sample in this format is 10 bits, encoded as a 16-bit integer in
    little-endian byte order.

    There are {{VideoFrame/codedWidth}} * {{VideoFrame/codedHeight}} samples
    in all three planes, arranged starting at the top left of the image, in
    {{VideoFrame/codedHeight}} rows of {{VideoFrame/codedWidth}} samples.
  </dd>
  <dt><dfn enum-value for=VideoPixelFormat>I444P12</dfn></dt>
  <dd>
    This format is composed of three distinct planes, one plane of Luma and two
    planes of Chroma, denoted Y, U and V, and present in this order.

    This format does not use [=sub-sampling=].

    Each sample in this format is 12 bits, encoded as a 16-bit integer in
    little-endian byte order.

    There are {{VideoFrame/codedWidth}} * {{VideoFrame/codedHeight}} samples
    in all three planes, arranged starting at the top left of the image, in
    {{VideoFrame/codedHeight}} rows of {{VideoFrame/codedWidth}} samples.
  </dd>
  <dt><dfn enum-value for=VideoPixelFormat>I444A</dfn></dt>
  <dd>
    This format is composed of four distinct planes, one plane of Luma, two
    planes of Chroma, denoted Y, U and V, and one plane of Alpha values, all
    present in this order.

    This format does not use [=sub-sampling=].

    Each sample in this format is 8 bits.

    There are {{VideoFrame/codedWidth}} * {{VideoFrame/codedHeight}} samples
    (and therefore bytes) in all four planes, arranged starting at the top left
    of the image, in {{VideoFrame/codedHeight}} rows of
    {{VideoFrame/codedWidth}} samples.

    {{I444A}}'s [=equivalent opaque format=] is {{I444}}.
  </dd>
  <dt><dfn enum-value for=VideoPixelFormat>I444AP10</dfn></dt>
  <dd>
    This format is composed of four distinct planes, one plane of Luma, two
    planes of Chroma, denoted Y, U and V, and one plane of Alpha values, all
    present in this order.

    This format does not use [=sub-sampling=].

    Each sample in this format is 10 bits, encoded as a 16-bit integer in
    little-endian byte order.

    There are {{VideoFrame/codedWidth}} * {{VideoFrame/codedHeight}} samples
    in all four planes, arranged starting at the top left of the image,
    in {{VideoFrame/codedHeight}} rows of {{VideoFrame/codedWidth}} samples.

    {{I444AP10}}'s [=equivalent opaque format=] is {{I444P10}}.
  </dd>
  <dt><dfn enum-value for=VideoPixelFormat>I444AP12</dfn></dt>
  <dd>
    This format is composed of four distinct planes, one plane of Luma, two
    planes of Chroma, denoted Y, U and V, and one plane of Alpha values, all
    present in this order.

    This format does not use [=sub-sampling=].

    Each sample in this format is 12 bits, encoded as a 16-bit integer in
    little-endian byte order.

    There are {{VideoFrame/codedWidth}} * {{VideoFrame/codedHeight}} samples
    in all four planes, arranged starting at the top left of the image,
    in {{VideoFrame/codedHeight}} rows of {{VideoFrame/codedWidth}} samples.

    {{I444AP10}}'s [=equivalent opaque format=] is {{I444P10}}.
  </dd>
  <dt><dfn enum-value for=VideoPixelFormat>NV12</dfn></dt>
  <dd>
    This format is composed of two distinct planes, one plane of Luma and then
    another plane for the two Chroma components. The two planes are present in
    this order, and are refered to as respectively the Y plane and the UV plane.

    The U and V components are [=sub-sampled=] horizontally and vertically by a
    [=factor=] of 2 compared to the components in the Y planes.

    Each sample in this format is 8 bits.

    There are {{VideoFrame/codedWidth}} * {{VideoFrame/codedHeight}} samples
    (and therefore bytes) in the Y and plane, arranged starting at the top left
    of the image, in {{VideoFrame/codedHeight}} rows of
    {{VideoFrame/codedWidth}} samples.

    The UV plane is composed of interleaved U and V values, in a number of
    rows equal to the result of the division of {{VideoFrame/codedHeight}}
    by 2, rounded up to the nearest integer. Each row has a number of elements
    equal to the result of the division of {{VideoFrame/codedWidth}} by 2,
    rounded up to the nearest integer. Each element is composed of two Chroma
    samples, the U and V samples, in that order. Samples are arranged starting
    at the top left of the image.

    The visible rectangle offset ({{VideoFrame/visibleRect}}.{{DOMRectInit/x}}
    and {{VideoFrame/visibleRect}}.{{DOMRectInit/y}})
    <em class="rfc2119">MUST</em> be even.

    <div class=example>
    An image in the NV12 pixel format that is 16 pixels wide and 10 pixels tall
    will be arranged like so in memory:

    ```
        YYYYYYYYYYYYYYYY
        YYYYYYYYYYYYYYYY
        YYYYYYYYYYYYYYYY
        YYYYYYYYYYYYYYYY
        YYYYYYYYYYYYYYYY
        YYYYYYYYYYYYYYYY
        YYYYYYYYYYYYYYYY
        YYYYYYYYYYYYYYYY
        YYYYYYYYYYYYYYYY
        YYYYYYYYYYYYYYYY
        UVUVUVUVUVUVUVUV
        UVUVUVUVUVUVUVUV
        UVUVUVUVUVUVUVUV
        UVUVUVUVUVUVUVUV
        UVUVUVUVUVUVUVUV
    ```

    All samples being linear in memory.
  </div>
  </dd>
  <dt><dfn enum-value for=VideoPixelFormat>RGBA</dfn></dt>
  <dd>
    This format is composed of a single plane, that encodes four components:
    Red, Green, Blue, and an alpha value, present in this order.

    Each sample in this format is 8 bits, and each pixel is therefore 32 bits.

    There are {{VideoFrame/codedWidth}} * {{VideoFrame/codedHeight}} * 4 samples
    (and therefore bytes) in the single plane, arranged starting at the top
    left of the image, in {{VideoFrame/codedHeight}} rows of
    {{VideoFrame/codedWidth}} samples.

    {{RGBA}}'s [=equivalent opaque format=] is {{RGBX}}.
  </dd>
  <dt><dfn enum-value for=VideoPixelFormat>RGBX</dfn></dt>
  <dd>
    This format is composed of a single plane, that encodes four components:
    Red, Green, Blue, and a padding value, present in this order.

    Each sample in this format is 8 bits. The fourth element in each pixel is to
    be ignored, the image is always fully opaque.

    There are {{VideoFrame/codedWidth}} * {{VideoFrame/codedHeight}} * 4 samples
    (and therefore bytes) in the single plane, arranged starting at the top left
    of the image, in {{VideoFrame/codedHeight}} rows of
    {{VideoFrame/codedWidth}} samples.
  </dd>
  <dt><dfn enum-value for=VideoPixelFormat>BGRA</dfn></dt>
  <dd>
    This format is composed of a single plane, that encodes four components:
    Blue, Green, Red, and an alpha value, present in this order.

    Each sample in this format is 8 bits.

    There are {{VideoFrame/codedWidth}} * {{VideoFrame/codedHeight}} * 4 samples
    (and therefore bytes) in the single plane, arranged starting at the top left
    of the image, in {{VideoFrame/codedHeight}} rows of
    {{VideoFrame/codedWidth}} samples.

    {{BGRA}}'s [=equivalent opaque format=] is {{BGRX}}.
  </dd>
  <dt><dfn enum-value for=VideoPixelFormat>BGRX</dfn></dt>
  <dd>
    This format is composed of a single plane, that encodes four components:
    Blue, Green, Red, and a padding value, present in this order.

    Each sample in this format is 8 bits. The fourth element in each pixel is to
    be ignored, the image is always fully opaque.

    There are {{VideoFrame/codedWidth}} * {{VideoFrame/codedHeight}} * 4 samples
    (and therefore bytes) in the single plane, arranged starting at the top left
    of the image, in {{VideoFrame/codedHeight}} rows of
    {{VideoFrame/codedWidth}} samples.
  </dd>
</dl>

Video Color Space Interface {#video-color-space-interface}
----------------------------------------------------------

<xmp class='idl'>
[Exposed=(Window,DedicatedWorker)]
interface VideoColorSpace {
  constructor(optional VideoColorSpaceInit init = {});

  readonly attribute VideoColorPrimaries? primaries;
  readonly attribute VideoTransferCharacteristics? transfer;
  readonly attribute VideoMatrixCoefficients? matrix;
  readonly attribute boolean? fullRange;

  [Default] VideoColorSpaceInit toJSON();
};

dictionary VideoColorSpaceInit {
  VideoColorPrimaries? primaries = null;
  VideoTransferCharacteristics? transfer = null;
  VideoMatrixCoefficients? matrix = null;
  boolean? fullRange = null;
};
</xmp>

### Internal Slots ### {#videocolorspace-internal-slots}
: <dfn attribute for=VideoColorSpace>\[[primaries]]</dfn>
:: The color primaries.
: <dfn attribute for=VideoColorSpace>\[[transfer]]</dfn>
:: The transfer characteristics.
: <dfn attribute for=VideoColorSpace>\[[matrix]]</dfn>
:: The matrix coefficients.
: <dfn attribute for=VideoColorSpace>[[full range]]</dfn>
:: Indicates whether full-range color values are used.

### Constructors ### {#videocolorspace-constructors}
<dfn constructor for=VideoColorSpace title="VideoColorSpace(init)">
  VideoColorSpace(init)
</dfn>
1. Let |c| be a new {{VideoColorSpace}} object, initialized as follows:
    1. Assign `init.primaries` to {{VideoColorSpace/[[primaries]]}}.
    2. Assign `init.transfer` to {{VideoColorSpace/[[transfer]]}}.
    3. Assign `init.matrix` to {{VideoColorSpace/[[matrix]]}}.
    4. Assign `init.fullRange` to {{VideoColorSpace/[[full range]]}}.
6. Return |c|.

### Attributes ### {#videocolorspace-attributes}
: <dfn attribute for=VideoColorSpace>primaries</dfn>
:: The {{VideoColorSpace/primaries}} getter steps are to return the value of
    {{VideoColorSpace/[[primaries]]}}.
: <dfn attribute for=VideoColorSpace>transfer</dfn>
:: The {{VideoColorSpace/transfer}} getter steps are to return the value of
    {{VideoColorSpace/[[transfer]]}}.
: <dfn attribute for=VideoColorSpace>matrix</dfn>
:: The {{VideoColorSpace/matrix}} getter steps are to return the value of
    {{VideoColorSpace/[[matrix]]}}.
: <dfn attribute for=VideoColorSpace>fullRange</dfn>
:: The {{VideoColorSpace/fullRange}} getter steps are to return the value of
    {{VideoColorSpace/[[full range]]}}.

Video Color Primaries {#videocolorprimaries}
--------------------------------------------
Color primaries describe the color gamut of video samples.

<xmp class='idl'>
enum VideoColorPrimaries {
  "bt709",
  "bt470bg",
  "smpte170m",
  "bt2020",
  "smpte432",
};
</xmp>

<dl>
  <dt><dfn enum-value for=VideoColorPrimaries>bt709</dfn></dt>
  <dd>
    Color primaries used by BT.709 and sRGB, as described by [[H.273]]
    section 8.1 table 2 value 1.
  </dd>
  <dt><dfn enum-value for=VideoColorPrimaries>bt470bg</dfn></dt>
  <dd>
    Color primaries used by BT.601 PAL, as described by [[H.273]]
    section 8.1 table 2 value 5.
  </dd>
  <dt><dfn enum-value for=VideoColorPrimaries>smpte170m</dfn></dt>
  <dd>
    Color primaries used by BT.601 NTSC, as described by [[H.273]]
    section 8.1 table 2 value 6.
  </dd>
  <dt><dfn enum-value for=VideoColorPrimaries>bt2020</dfn></dt>
  <dd>
    Color primaries used by BT.2020 and BT.2100, as described by [[H.273]]
    section 8.1 table 2 value 9.
  </dd>
  <dt><dfn enum-value for=VideoColorPrimaries>smpte432</dfn></dt>
  <dd>
    Color primaries used by P3 D65, as described by [[H.273]]
    section 8.1 table 2 value 12.
  </dd>
</dl>

Video Transfer Characteristics {#videotransfercharacteristics}
--------------------------------------------------------------
Transfer characteristics describe the opto-electronic transfer characteristics
of video samples.

<xmp class='idl'>
enum VideoTransferCharacteristics {
  "bt709",
  "smpte170m",
  "iec61966-2-1",
  "linear",
  "pq",
  "hlg",
};
</xmp>

<dl>
  <dt><dfn enum-value for=VideoTransferCharacteristics>bt709</dfn></dt>
  <dd>
    Transfer characteristics used by BT.709, as described by [[H.273]]
    section 8.2 table 3 value 1.
  </dd>
  <dt><dfn enum-value for=VideoTransferCharacteristics>smpte170m</dfn></dt>
  <dd>
    Transfer characteristics used by BT.601, as described by [[H.273]]
    section 8.2 table 3 value 6. (Functionally the same as "bt709".)
  </dd>
  <dt><dfn enum-value for=VideoTransferCharacteristics>iec61966-2-1</dfn></dt>
  <dd>
    Transfer characteristics used by sRGB, as described by [[H.273]]
    section 8.2 table 3 value 13.
  </dd>
  <dt><dfn enum-value for=VideoTransferCharacteristics>linear</dfn></dt>
  <dd>
    Transfer characteristics used by linear RGB, as described by [[H.273]]
    section 8.2 table 3 value 8.
  </dd>
  <dt><dfn enum-value for=VideoTransferCharacteristics>pq</dfn></dt>
  <dd>
    Transfer characteristics used by BT.2100 PQ, as described by [[H.273]]
    section 8.2 table 3 value 16.
  </dd>
  <dt><dfn enum-value for=VideoTransferCharacteristics>hlg</dfn></dt>
  <dd>
    Transfer characteristics used by BT.2100 HLG, as described by [[H.273]]
    section 8.2 table 3 value 18.
  </dd>
</dl>

Video Matrix Coefficients {#videomatrixcoefficients}
----------------------------------------------------
Matrix coefficients describe the relationship between sample component values
and color coordinates.

<xmp class='idl'>
enum VideoMatrixCoefficients {
  "rgb",
  "bt709",
  "bt470bg",
  "smpte170m",
  "bt2020-ncl",
};
</xmp>

<dl>
  <dt><dfn enum-value for=VideoMatrixCoefficients>rgb</dfn></dt>
  <dd>
    Matrix coefficients used by sRGB, as described by [[H.273]]
    section 8.3 table 4 value 0.
  </dd>
  <dt><dfn enum-value for=VideoMatrixCoefficients>bt709</dfn></dt>
  <dd>
    Matrix coefficients used by BT.709, as described by [[H.273]]
    section 8.3 table 4 value 1.
  </dd>
  <dt><dfn enum-value for=VideoMatrixCoefficients>bt470bg</dfn></dt>
  <dd>
    Matrix coefficients used by BT.601 PAL, as described by [[H.273]]
    section 8.3 table 4 value 5.
  </dd>
  <dt><dfn enum-value for=VideoMatrixCoefficients>smpte170m</dfn></dt>
  <dd>
    Matrix coefficients used by BT.601 NTSC, as described by [[H.273]]
    section 8.3 table 4 value 6. (Functionally the same as "bt470bg".)
  </dd>
  <dt><dfn enum-value for=VideoMatrixCoefficients>bt2020-ncl</dfn></dt>
  <dd>
    Matrix coefficients used by BT.2020 NCL, as described by [[H.273]]
    section 8.3 table 4 value 9.
  </dd>
</dl>


Image Decoding {#image-decoding}
====================================

Background {#image-decoding-background}
-------------------------------------

<div class=non-normative>
  This section is non-normative.

  Image codec definitions are typically accompanied by a definition for a
  corresponding file format. Hence image decoders often perform both duties of
  unpacking (demuxing) as well as decoding the encoded image data. The WebCodecs
  {{ImageDecoder}} follows this pattern, which motivates an interface design that
  is notably different from that of {{VideoDecoder}} and {{AudioDecoder}}.

  In spite of these differences, {{ImageDecoder}} uses the same
  [=codec processing model=] as the other codec interfaces. Additionally,
  {{ImageDecoder}} uses the {{VideoFrame}} interface to describe decoded outputs.
</div>

ImageDecoder Interface {#imagedecoder-interface}
------------------------------------------------

<pre class='idl'>
<xmp>
[Exposed=(Window,DedicatedWorker), SecureContext]
interface ImageDecoder {
  constructor(ImageDecoderInit init);

  readonly attribute DOMString type;
  readonly attribute boolean complete;
  readonly attribute Promise<undefined> completed;
  readonly attribute ImageTrackList tracks;

  Promise<ImageDecodeResult> decode(optional ImageDecodeOptions options = {});
  undefined reset();
  undefined close();

  static Promise<boolean> isTypeSupported(DOMString type);
};
</xmp>
</pre>

### Internal Slots ### {#imagedecoder-internal-slots}

: <dfn attribute for=ImageDecoder>[[control message queue]]</dfn>
:: A [=queue=] of [=control messages=] to be performed upon this [=codec=]
    instance. See [=[[control message queue]]=].

: <dfn attribute for=ImageDecoder>[[message queue blocked]]</dfn>
:: A boolean indicating when processing the
    {{ImageDecoder/[[control message queue]]}} is blocked by a pending
    [=control message=]. See [=[[message queue blocked]]=].

: <dfn attribute for=ImageDecoder>[[codec work queue]]</dfn>
:: A [=parallel queue=] used for running parallel steps that reference the
    {{ImageDecoder/[[codec implementation]]}}. See [=[[codec work queue]]=].

: <dfn attribute for=ImageDecoder>\[[ImageTrackList]]</dfn>
:: An {{ImageTrackList}} describing the tracks found in
    {{ImageDecoder/[[encoded data]]}}

: <dfn attribute for=ImageDecoder>\[[type]]</dfn>
:: A string reflecting the value of the MIME {{ImageDecoderInit/type}} given at
    construction.

: <dfn attribute for=ImageDecoder>\[[complete]]</dfn>
:: A boolean indicating whether {{ImageDecoder/[[encoded data]]}} is completely
    buffered.

: <dfn attribute for=ImageDecoder>[[completed promise]]</dfn>
:: The promise used to signal when {{ImageDecoder/[[complete]]}} becomes
    `true`.

: <dfn attribute for=ImageDecoder>[[codec implementation]]</dfn>
:: An underlying image decoder implementation provided by the User Agent. See
    [=[[codec implementation]]=].

: <dfn attribute for=ImageDecoder>[[encoded data]]</dfn>
:: A [=byte sequence=] containing the encoded image data to be decoded.

: <dfn attribute for=ImageDecoder>[[prefer animation]]</dfn>
:: A boolean reflecting the value of {{ImageDecoderInit/preferAnimation}} given
    at construction.

: <dfn attribute for=ImageDecoder>[[pending decode promises]]</dfn>
:: A list of unresolved promises returned by calls to decode().

: <dfn attribute for=ImageDecoder>[[internal selected track index]]</dfn>
:: Identifies the image track within {{ImageDecoder/[[encoded data]]}} that is
    used by decoding algorithms.

: <dfn attribute for=ImageDecoder>[[tracks established]]</dfn>
:: A boolean indicating whether the track list has been established in
    {{ImageDecoder/[[ImageTrackList]]}}.

: <dfn attribute for=ImageDecoder>\[[closed]]</dfn>
:: A boolean indicating that the {{ImageDecoder}} is in a permanent closed state
    and can no longer be used.

: <dfn attribute for=ImageDecoder>[[progressive frame generations]]</dfn>
:: A mapping of frame indices to [=Progressive Image Frame Generations=]. The
    values represent the Progressive Image Frame Generation for the
    {{VideoFrame}} which was most recently output by a call to
    {{ImageDecoder/decode()}} with the given frame index.


### Constructor ### {#imagedecoder-constructor}

: <dfn constructor for=ImageDecoder title="ImageDecoder(init)">
    ImageDecoder(init)
    </dfn>
:: NOTE: Calling {{ImageDecoder/decode()}} on the constructed {{ImageDecoder}}
    will trigger a {{NotSupportedError}} if the User Agent does not support
    |type|. Authors are encouraged to first check support by calling
    {{ImageDecoder/isTypeSupported()}} with |type|. User Agents don't have to
    support any particular type.

    When invoked, run these steps:
    1. If |init| is not [=valid ImageDecoderInit=], throw a {{TypeError}}.
    2. If |init|.{{ImageDecoderInit/transfer}} contains more than one reference
        to the same {{ArrayBuffer}}, then throw a {{DataCloneError}} {{DOMException}}.
    3. For each |transferable| in |init|.{{ImageDecoderInit/transfer}}:
        1. If {{platform object/[[Detached]]}} internal slot is `true`,
            then throw a {{DataCloneError}} {{DOMException}}.
    4. Let |d| be a new {{ImageDecoder}} object. In the steps below, all
        mentions of {{ImageDecoder}} members apply to |d| unless stated
        otherwise.
    5. Assign a new [=queue=] to {{ImageDecoder/[[control message queue]]}}.
    6. Assign `false` to {{ImageDecoder/[[message queue blocked]]}}.
    7. Assign the result of starting a new [=parallel queue=] to
        {{ImageDecoder/[[codec work queue]]}}.
    8. Assign {{ImageDecoder/[[ImageTrackList]]}} a new {{ImageTrackList}}
        initialized as follows:
        1. Assign a new [=list=] to {{ImageTrackList/[[track list]]}}.
        2. Assign `-1` to {{ImageTrackList/[[selected index]]}}.
        3. Assign a new promise to {{ImageTrackList/[[ready promise]]}} and [=mark as handled|mark it as handled=].
    9. Assign {{ImageDecoderInit/type}} to {{ImageDecoder/[[type]]}}.
    10. Assign `null` to {{ImageDecoder/[[codec implementation]]}}.
    11. If `init.preferAnimation` [=map/exists=], assign `init.preferAnimation`
        to the {{ImageDecoder/[[prefer animation]]}} internal slot. Otherwise,
        assign 'null' to {{ImageDecoder/[[prefer animation]]}} internal slot.
    12. Assign a new promise to {{ImageDecoder/[[completed promise]]}} and [=mark as handled|mark it as handled=].
    13. Assign a new [=list=] to {{ImageDecoder/[[pending decode promises]]}}.
    14. Assign `-1` to {{ImageDecoder/[[internal selected track index]]}}.
    15. Assign `false` to {{ImageDecoder/[[tracks established]]}}.
    16. Assign `false` to {{ImageDecoder/[[closed]]}}.
    17. Assign a new [=map=] to {{ImageDecoder/[[progressive frame
        generations]]}}.
    17. If |init|'s {{ImageDecoderInit/data}} member is of type
        {{ReadableStream}}:
        1. Assign a new [=list=] to {{ImageDecoder/[[encoded data]]}}.
        2. Assign `false` to {{ImageDecoder/[[complete]]}}
        3. [=Queue a control message=] to [=configure the image decoder=] with
            |init|.
        4. [=Process the control message queue=].
        5. Let |reader| be the result of [=getting a reader=] for
            {{ImageDecoderInit/data}}.
        6. In parallel, perform the [=Fetch Stream Data Loop=] on |d| with
            |reader|.
    18. Otherwise:
        1. Assert that `init.data` is of type {{BufferSource}}.
        2. If |init|.{{ImageDecoderInit/transfer}} contains an {{ArrayBuffer}}
            referenced by |init|.{{ImageDecoderInit/data}} the User Agent
            <em class="rfc2119">MAY</em> choose to:
            1. Let {{ImageDecoder/[[encoded data]]}} reference bytes in |data|
                representing an encoded image.
        3. Otherwise:
            1. Assign a copy of `init.data` to  {{ImageDecoder/[[encoded data]]}}.
        4. Assign `true` to {{ImageDecoder/[[complete]]}}.
        5. Resolve {{ImageDecoder/[[completed promise]]}}.
        6. Queue a control message to [=configure the image decoder=] with
            |init|.
        7. Queue a control message to [=decode track metadata=].
        8. [=Process the control message queue=].
    19. For each |transferable| in |init|.{{ImageDecoderInit/transfer}}:
        1. Perform [DetachArrayBuffer](https://tc39.es/ecma262/#sec-detacharraybuffer)
            on |transferable|
    20. return |d|.

    [=Running a control message=] to <dfn>configure the image decoder</dfn>
    means running these steps:
    1. Let |supported| be the result of running the [=ImageDecoder/Check Type
        Support=] algorithm with `init.type`.
    2. If |supported| is `false`, run the [=ImageDecoder/Close ImageDecoder=]
        algorithm with a {{NotSupportedError}} {{DOMException}} and return
        `"processed"`.
    3. Otherwise, assign the {{ImageDecoder/[[codec implementation]]}} internal
        slot with an implementation supporting `init.type`
    4. Assign `true` to {{ImageDecoder/[[message queue blocked]]}}.
    5. Enqueue the following steps to the {{ImageDecoder/[[codec work queue]]}}:
        1. Configure {{ImageDecoder/[[codec implementation]]}} in accordance
            with the values given for {{ImageDecoderInit/colorSpaceConversion}},
            {{ImageDecoderInit/desiredWidth}}, and
            {{ImageDecoderInit/desiredHeight}}.
        2. Assign `false` to {{ImageDecoder/[[message queue blocked]]}}.
        3. [=Queue a task=] to [=Process the control message queue=].
    6. Return `"processed"`.

    [=Running a control message=] to <dfn>decode track metadata</dfn> means
    running these steps:
    1. Enqueue the following steps to the {{ImageDecoder/[[codec work queue]]}}:
        1. Run the [=ImageDecoder/Establish Tracks=] algorithm.

### Attributes ### {#imagedecoder-attributes}
: <dfn attribute for=ImageDecoder>type</dfn>
:: A string reflecting the value of the MIME {{ImageDecoderInit/type}} given at
    construction.

    The {{ImageDecoder/type}} getter steps are to return
    {{ImageDecoder/[[type]]}}.

: <dfn attribute for=ImageDecoder>complete</dfn>
:: Indicates whether {{ImageDecoder/[[encoded data]]}} is completely buffered.

    The {{ImageDecoder/complete}} getter steps are to return
    {{ImageDecoder/[[complete]]}}.

: <dfn attribute for=ImageDecoder>completed</dfn>
:: The promise used to signal when {{ImageDecoder/complete}} becomes `true`.

    The {{ImageDecoder/completed}} getter steps are to return
    {{ImageDecoder/[[completed promise]]}}.

: <dfn attribute for=ImageDecoder>tracks</dfn>
:: Returns a [=live=] {{ImageTrackList}}, which provides metadata
    for the available tracks and a mechanism for selecting a track to decode.

    The {{ImageDecoder/tracks}} getter steps are to return
    {{ImageDecoder/[[ImageTrackList]]}}.

### Methods ### {#imagedecoder-methods}
: <dfn method for=ImageDecoder>decode(options)</dfn>
:: Enqueues a control message to decode the frame according to |options|.

    When invoked, run these steps:
    1. If {{ImageDecoder/[[closed]]}} is `true`, return a {{Promise}}
        rejected with an {{InvalidStateError}} {{DOMException}}.
    2. If {{ImageDecoder/[[ImageTrackList]]}}'s
        {{ImageTrackList/[[selected index]]}} is '-1', return a {{Promise}}
        rejected with an {{InvalidStateError}} {{DOMException}}.
    3. If |options| is `undefined`, assign a new {{ImageDecodeOptions}} to
        |options|.
    4. Let |promise| be a new {{Promise}}.
    5. Append |promise| to {{ImageDecoder/[[pending decode promises]]}}.
    6. [=Queue a control message=] to decode the image with |options|, and
        |promise|.
    7. [=Process the control message queue=].
    8. Return |promise|.

    [=Running a control message=] to decode the image means running these
    steps:
    1. Enqueue the following steps to the {{ImageDecoder/[[codec work queue]]}}:
        1. Wait for {{ImageDecoder/[[tracks established]]}} to become `true`.
        2. If |options|.{{ImageDecodeOptions/completeFramesOnly}} is `false` and
            the image is a [=Progressive Image=] for which the User Agent
            supports progressive decoding, run the [=Decode Progressive Frame=]
            algorithm with |options|.{{ImageDecodeOptions/frameIndex}} and
            |promise|.
        3. Otherwise, run the [=Decode Complete Frame=] algorithm with
            |options|.{{ImageDecodeOptions/frameIndex}} and |promise|.

: <dfn method for=ImageDecoder>reset()</dfn>
:: Immediately aborts all pending work.

    When invoked, run the [=ImageDecoder/Reset ImageDecoder=] algorithm with
    an {{AbortError}} {{DOMException}}.

: <dfn method for=ImageDecoder>close()</dfn>
:: Immediately aborts all pending work and releases system resources. Close is
    final.

    When invoked, run the [=ImageDecoder/Close ImageDecoder=] algorithm with
    an {{AbortError}} {{DOMException}}.

: <dfn method for=ImageDecoder>isTypeSupported(type)</dfn>
:: Returns a promise indicating whether the provided config is supported by the
    User Agent.

    When invoked, run these steps:
    1. If |type| is not a [=valid image MIME type=], return a {{Promise}}
        rejected with {{TypeError}}.
    2. Let |p| be a new {{Promise}}.
    3. Run the following steps [=in parallel=]:
        1. Let |isSupported| be the result of running the
            [=Check Type Support=] algorithm with |type|.
        2. [=Queue a task=] to resolve |p| with |isSupported|.
    4. Return |p|.

### Algorithms ### {#imagedecoder-algorithms}

: <dfn for=ImageDecoder>Fetch Stream Data Loop</dfn> (with |reader|)
:: Run these steps:
    1. Let |readRequest| be the following [=read request=].

        : [=read request/chunk steps=], given |chunk|
        :: 1. If {{ImageDecoder/[[closed]]}} is `true`, abort these steps.
            2. If |chunk| is not a Uint8Array object, [=queue a task=] to run
                the [=ImageDecoder/Close ImageDecoder=] algorithm with a
                {{DataError}} {{DOMException}} and abort these steps.
            3. Let |bytes| be the byte sequence represented by the Uint8Array
                object.
            4. Append |bytes| to the  {{ImageDecoder/[[encoded data]]}}
                internal slot.
            5. If {{ImageDecoder/[[tracks established]]}} is `false`, run the
                [=Establish Tracks=] algorithm.
            6. Otherwise, run the [=Update Tracks=] algorithm.
            7. Run the [=Fetch Stream Data Loop=] algorithm with |reader|.

        : [=read request/close steps=]
        :: 1. Assign `true` to {{ImageDecoder/[[complete]]}}
            2. Resolve {{ImageDecoder/[[completed promise]]}}.

        : [=read request/error steps=]
        :: 1. [=Queue a task=] to run the [=ImageDecoder/Close ImageDecoder=]
                algorithm with a {{NotReadableError}} {{DOMException}}

    2. Read a chunk from |reader| given |readRequest|.

: <dfn for=ImageDecoder>Establish Tracks</dfn>
:: Run these steps:
    1. Assert {{ImageDecoder/[[tracks established]]}} is `false`.
    2. If {{ImageDecoder/[[encoded data]]}} does not contain enough data to
        determine the number of tracks:
        1. If {{ImageDecoder/complete}} is `true`, [=queue a task=] to run the
            [=ImageDecoder/Close ImageDecoder=] algorithm with a
            {{InvalidStateError}} {{DOMException}}.
        2. Abort these steps.
    3. If the number of tracks is found to be `0`, [=queue a task=] to run the
        [=ImageDecoder/Close ImageDecoder=] algorithm and abort these steps.
    4. Let |newTrackList| be a new [=list=].
    5. For each |image track| found in {{ImageDecoder/[[encoded data]]}}:
        1. Let |newTrack| be a new {{ImageTrack}}, initialized as follows:
            1. Assign [=this=] to {{ImageTrack/[[ImageDecoder]]}}.
            2. Assign {{ImageDecoder/tracks}} to
                {{ImageTrack/[[ImageTrackList]]}}.
            3. If |image track| is found to be animated, assign `true` to
                |newTrack|'s {{ImageTrack/[[animated]]}} internal slot.
                Otherwise, assign `false`.
            4. If |image track| is found to describe a frame count, assign
                that count to |newTrack|'s {{ImageTrack/[[frame count]]}}
                internal slot. Otherwise, assign `0`.

                NOTE: If [=this=] was constructed with
                  {{ImageDecoderInit/data}} as a {{ReadableStream}}, the
                  {{ImageTrack/frameCount}} can change as additional bytes are
                  appended to {{ImageDecoder/[[encoded data]]}}. See the
                  [=Update Tracks=] algorithm.

            5. If |image track| is found to describe a repetition count,
                assign that count to {{ImageTrack/[[repetition count]]}}
                internal slot. Otherwise, assign `0`.

                NOTE: A value of `Infinity` indicates infinite repetitions.

            6. Assign `false` to |newTrack|'s {{ImageTrack/[[selected]]}}
                internal slot.
        2. Append |newTrack| to |newTrackList|.
    6. Let |selectedTrackIndex| be the result of running the
        [=ImageDecoder/Get Default Selected Track Index=] algorithm with
        |newTrackList|.
    7. Let |selectedTrack| be the track at position |selectedTrackIndex| within
        |newTrackList|.
    8. Assign `true` to |selectedTrack|'s {{ImageTrack/[[selected]]}} internal
        slot.
    8. Assign |selectedTrackIndex| to {{ImageDecoder/[[internal selected track
        index]]}}.
    9. Assign `true` to {{ImageDecoder/[[tracks established]]}}.
    10. [=Queue a task=] to perform the following steps:
        1. Assign |newTrackList| to the {{ImageDecoder/tracks}}
            {{ImageTrackList/[[track list]]}} internal slot.
        2. Assign |selectedTrackIndex| to {{ImageDecoder/tracks}}
            {{ImageTrackList/[[selected index]]}}.
        3. Resolve {{ImageTrackList/[[ready promise]]}}.

: <dfn for=ImageDecoder>Get Default Selected Track Index</dfn> (with
    |trackList|)
:: Run these steps:
    1. If {{ImageDecoder/[[encoded data]]}} identifies a [=Primary Image
        Track=]:
        1. Let |primaryTrack| be the {{ImageTrack}} from |trackList| that
            describes the [=Primary Image Track=].
        2. Let |primaryTrackIndex| be position of |primaryTrack| within
            |trackList|.
        3. If {{ImageDecoder/[[prefer animation]]}} is `null`, return
            |primaryTrackIndex|.
        4. If |primaryTrack|.{{ImageTrack/animated}} equals
            {{ImageDecoder/[[prefer animation]]}}, return |primaryTrackIndex|.
    2. If any {{ImageTrack}}s in |trackList| have {{ImageTrack/animated}} equal
        to {{ImageDecoder/[[prefer animation]]}}, return the position of the
        earliest such track in |trackList|.
    3. Return `0`.

: <dfn for=ImageDecoder>Update Tracks</dfn>
:: A <dfn>track update struct</dfn> is a [=struct=] that consists of a
    <dfn for="track update struct">track index</dfn> ({{unsigned long}})
    and a <dfn for="track update struct">frame count</dfn>
    ({{unsigned long}}).

    Run these steps:
    1. Assert {{ImageDecoder/[[tracks established]]}} is `true`.
    2. Let |trackChanges| be a new [=list=].
    3. Let |trackList| be a copy of {{ImageDecoder/tracks}}'
        {{ImageTrackList/[[track list]]}}.
    4. For each |track| in |trackList|:
        1. Let |trackIndex| be  the position of |track| in |trackList|.
        2. Let |latestFrameCount| be the frame count as indicated by
            {{ImageDecoder/[[encoded data]]}} for the track corresponding to
            |track|.
        3. Assert that |latestFrameCount| is greater than or equal to
            `track.frameCount`.
        4. If |latestFrameCount| is greater than `track.frameCount`:
            1. Let |change| be a [=track update struct=] whose
                [=track update struct/track index=] is |trackIndex| and
                [=track update struct/frame count=] is |latestFrameCount|.
            2. Append |change| to |tracksChanges|.
    5. If |tracksChanges| [=list/is empty=], abort these steps.
    6. [=Queue a task=] to perform the following steps:
        1. For each <var ignore=''>update</var> in |trackChanges|:
            1. Let |updateTrack| be the {{ImageTrack}} at position
                `update.trackIndex` within {{ImageDecoder/tracks}}'
                {{ImageTrackList/[[track list]]}}.
            2. Assign `update.frameCount` to |updateTrack|'s
                {{ImageTrack/[[frame count]]}}.

: <dfn for=ImageDecoder>Decode Complete Frame</dfn> (with |frameIndex| and
    |promise|)
:: 1. Assert that {{ImageDecoder/[[tracks established]]}} is `true`.
    2. Assert that {{ImageDecoder/[[internal selected track index]]}} is not
        `-1`.
    3. Let |encodedFrame| be the encoded frame identified by |frameIndex| and
        {{ImageDecoder/[[internal selected track index]]}}.
    4. Wait for any of the following conditions to be true (whichever happens
        first):
        1. {{ImageDecoder/[[encoded data]]}} contains enough bytes to
            completely decode |encodedFrame|.
        2. {{ImageDecoder/[[encoded data]]}} is found to be malformed.
        3. {{ImageDecoder/complete}} is `true`.
        4. {{ImageDecoder/[[closed]]}} is `true`.
    5. If {{ImageDecoder/[[encoded data]]}} is found to be malformed, run the
        [=ImageDecoder/Fatally Reject Bad Data=] algorithm and abort these
        steps.
    6. If {{ImageDecoder/[[encoded data]]}} does not contain enough bytes to
        completely decode |encodedFrame|, run the
        [=ImageDecoder/Reject Infeasible Decode=] algorithm with |promise| and
        abort these steps.
    7. Attempt to use {{ImageDecoder/[[codec implementation]]}} to decode
        |encodedFrame|.
    8. If decoding produces an error, run the
        [=ImageDecoder/Fatally Reject Bad Data=] algorithm and abort these
        steps.
    9. If {{ImageDecoder/[[progressive frame generations]]}} contains an entry
        keyed by |frameIndex|, remove the entry from the map.
    10. Let |output| be the decoded image data emitted by
        {{ImageDecoder/[[codec implementation]]}} corresponding to
        |encodedFrame|.
    11. Let |decodeResult| be a new {{ImageDecodeResult}} initialized as
        follows:
        1. Assign 'true' to {{ImageDecodeResult/complete}}.
        2. Let |duration| be the presentation duration for |output| as
            described by |encodedFrame|. If |encodedFrame| does not have a
            duration, assign `null` to |duration|.
        3. Let |timestamp| be the presentation timestamp for |output| as
            described by |encodedFrame|. If |encodedFrame| does not have a
            timestamp:
                1. If |encodedFrame| is a still image assign `0` to |timestamp|.
                2. If |encodedFrame| is a constant rate animated image and
                    |duration| is not `null`, assign `|frameIndex| * |duration|` to
                    |timestamp|.
                3. If a |timestamp| can otherwise be trivially generated from
                    metadata without further decoding, assign that to |timestamp|.
                4. Otherwise, assign `0` to |timestamp|.
        4. If {{ImageDecoder/[[encoded data]]}} contains orientation metadata
            describe it as |rotation| and |flip|, otherwise set |rotation| to 0
            and |flip| to false.
        5. Assign {{ImageDecodeResult/image}} with the result of running the
            [=Create a VideoFrame=] algorithm with |output|, |timestamp|,
            |duration|, |rotation|, and |flip|.
    12. Run the [=ImageDecoder/Resolve Decode=] algorithm with |promise| and
        |decodeResult|.

: <dfn for=ImageDecoder>Decode Progressive Frame</dfn> (with |frameIndex| and
    |promise|)
:: 1. Assert that {{ImageDecoder/[[tracks established]]}} is `true`.
    2. Assert that {{ImageDecoder/[[internal selected track index]]}} is not
        `-1`.
    3. Let |encodedFrame| be the encoded frame identified by |frameIndex| and
        {{ImageDecoder/[[internal selected track index]]}}.
    4. Let |lastFrameGeneration| be `null`.
    5. If {{ImageDecoder/[[progressive frame generations]]}} contains a map
        entry with the key |frameIndex|, assign the value of the map entry to
        |lastFrameGeneration|.
    6. Wait for any of the following conditions to be true (whichever happens
        first):
        1. {{ImageDecoder/[[encoded data]]}} contains enough bytes to decode
            |encodedFrame| to produce an output whose [=Progressive Image
            Frame Generation=] exceeds |lastFrameGeneration|.
        2. {{ImageDecoder/[[encoded data]]}} is found to be malformed.
        3. {{ImageDecoder/complete}} is `true`.
        4. {{ImageDecoder/[[closed]]}} is `true`.
    7. If {{ImageDecoder/[[encoded data]]}} is found to be malformed, run the
        [=ImageDecoder/Fatally Reject Bad Data=] algorithm and abort these
        steps.
    8. Otherwise, if {{ImageDecoder/[[encoded data]]}} does not contain enough
        bytes to decode |encodedFrame| to produce an output whose
        [=Progressive Image Frame Generation=] exceeds |lastFrameGeneration|,
        run the [=ImageDecoder/Reject Infeasible Decode=] algorithm with
        |promise| and abort these steps.
    9. Attempt to use {{ImageDecoder/[[codec implementation]]}} to decode
        |encodedFrame|.
    10. If decoding produces an error, run the
        [=ImageDecoder/Fatally Reject Bad Data=] algorithm and abort these
        steps.
    11. Let |output| be the decoded image data emitted by
        {{ImageDecoder/[[codec implementation]]}} corresponding to
        |encodedFrame|.
    12. Let |decodeResult| be a new {{ImageDecodeResult}}.
    13. If |output| is the final full-detail progressive output corresponding
        to |encodedFrame|:
        1. Assign `true` to |decodeResult|'s {{ImageDecodeResult/complete}}.
        2. If {{ImageDecoder/[[progressive frame generations]]}} contains an
            entry keyed by |frameIndex|, remove the entry from the map.
    14. Otherwise:
        1. Assign `false` to |decodeResult|'s {{ImageDecodeResult/complete}}.
        2. Let |frameGeneration| be the [=Progressive Image Frame Generation=]
            for |output|.
        3. Add a new entry to {{ImageDecoder/[[progressive frame
            generations]]}} with key |frameIndex| and value |frameGeneration|.
    15. Let |duration| be the presentation duration for |output| as
        described by |encodedFrame|. If |encodedFrame| does not describe a
        duration, assign `null` to |duration|.
    16. Let |timestamp| be the presentation timestamp for |output| as
        described by |encodedFrame|. If |encodedFrame| does not have a
        timestamp:
            1. If |encodedFrame| is a still image assign `0` to |timestamp|.
            2. If |encodedFrame| is a constant rate animated image and
                |duration| is not `null`, assign `|frameIndex| * |duration|` to
                |timestamp|.
            3. If a |timestamp| can otherwise be trivially generated from
                metadata without further decoding, assign that to |timestamp|.
            4. Otherwise, assign `0` to |timestamp|.
    17. If {{ImageDecoder/[[encoded data]]}} contains orientation metadata
        describe it as |rotation| and |flip|, otherwise set |rotation| to 0
        and |flip| to false.
    18. Assign {{ImageDecodeResult/image}} with the result of running the
        [=Create a VideoFrame=] algorithm with |output|, |timestamp|,
        |duration|, |rotation|, and |flip|.
    19. Remove |promise| from {{ImageDecoder/[[pending decode promises]]}}.
    20. Resolve |promise| with |decodeResult|.

: <dfn for=ImageDecoder>Resolve Decode</dfn> (with |promise| and |result|)
:: 1. [=Queue a task=] to perform these steps:
        1. If {{ImageDecoder/[[closed]]}}, abort these steps.
        2. Assert that |promise| is an element of
            {{ImageDecoder/[[pending decode promises]]}}.
        3. Remove |promise| from {{ImageDecoder/[[pending decode promises]]}}.
        4. Resolve |promise| with |result|.

: <dfn for=ImageDecoder>Reject Infeasible Decode</dfn> (with |promise|)
:: 1. Assert that {{ImageDecoder/complete}} is `true` or
        {{ImageDecoder/[[closed]]}} is `true`.
    2. If {{ImageDecoder/complete}} is `true`, let |exception| be a
            {{RangeError}}. Otherwise, let |exception| be an
            {{InvalidStateError}} {{DOMException}}.
    3. [=Queue a task=] to perform these steps:
        1. If {{ImageDecoder/[[closed]]}}, abort these steps.
        2. Assert that |promise| is an element of
            {{ImageDecoder/[[pending decode promises]]}}.
        3. Remove |promise| from {{ImageDecoder/[[pending decode promises]]}}.
        4. Reject |promise| with |exception|.

: <dfn for=ImageDecoder>Fatally Reject Bad Data</dfn>
:: 1. [=Queue a task=] to perform these steps:
        1. If {{ImageDecoder/[[closed]]}}, abort these steps.
        2. Run the [=ImageDecoder/Close ImageDecoder=] algorithm with an
            {{EncodingError}} {{DOMException}}.

: <dfn for=ImageDecoder>Check Type Support</dfn> (with |type|)
:: 1. If the User Agent can provide a codec to support decoding |type|, return
        `true`.
    2. Otherwise, return `false`.

: <dfn for=ImageDecoder>Reset ImageDecoder</dfn> (with |exception|)
:: 1. Signal {{ImageDecoder/[[codec implementation]]}} to abort any active
        decoding operation.
    2. For each |decodePromise| in
        {{ImageDecoder/[[pending decode promises]]}}:
        1. Reject |decodePromise| with |exception|.
        2. Remove |decodePromise| from
            {{ImageDecoder/[[pending decode promises]]}}.

: <dfn for=ImageDecoder>Close ImageDecoder</dfn> (with |exception|)
:: 1. Run the [=ImageDecoder/Reset ImageDecoder=] algorithm with |exception|.
    1. Assign `true` to {{ImageDecoder/[[closed]]}}.
    2. Clear {{ImageDecoder/[[codec implementation]]}} and release associated
        [=system resources=].
    3. If {{ImageDecoder/[[ImageTrackList]]}} is empty, reject
        {{ImageTrackList/[[ready promise]]}} with |exception|. Otherwise
        perform these steps,
        1. Remove all entries from {{ImageDecoder/[[ImageTrackList]]}}.
        2. Assign `-1` to {{ImageDecoder/[[ImageTrackList]]}}'s
            {{ImageTrackList/[[selected index]]}}.
    4. If {{ImageDecoder/[[complete]]}} is false resolve
        {{ImageDecoder/[[completed promise]]}} with |exception|.

ImageDecoderInit Interface {#imagedecoderinit-interface}
--------------------------------------------------------
<pre class='idl'>
<xmp>
typedef (AllowSharedBufferSource or ReadableStream) ImageBufferSource;
dictionary ImageDecoderInit {
  required DOMString type;
  required ImageBufferSource data;
  ColorSpaceConversion colorSpaceConversion = "default";
  [EnforceRange] unsigned long desiredWidth;
  [EnforceRange] unsigned long desiredHeight;
  boolean preferAnimation;
  sequence<ArrayBuffer> transfer = [];
};
</xmp>
</pre>

To determine if an {{ImageDecoderInit}} is a <dfn>valid ImageDecoderInit</dfn>,
run these steps:
1. If |type| is not a [=valid image MIME type=], return `false`.
2. If |data| is of type {{ReadableStream}} and the ReadableStream is
    [=ReadableStream/disturbed=] or [=ReadableStream/locked=], return `false`.
3. If |data| is of type {{BufferSource}}:
    1. If |data| is [[=BufferSource/detached=]], return false.
    2. If |data| [=list/is empty=], return `false`.
4. If {{ImageDecoderInit/desiredWidth}} [=map/exists=] and
    {{ImageDecoderInit/desiredHeight}} does not exist, return `false`.
5. If {{ImageDecoderInit/desiredHeight}} [=map/exists=] and
    {{ImageDecoderInit/desiredWidth}} does not exist, return `false`.
6. Return `true`.

A <dfn>valid image MIME type</dfn> is a string that is a [=valid MIME type
string=] and for which the `type`, per Section 8.3.1 of [[RFC9110]], is
`image`.

: <dfn dict-member for=ImageDecoderInit>type</dfn>
:: String containing the MIME type of the image file to be decoded.

: <dfn dict-member for=ImageDecoderInit>data</dfn>
:: {{BufferSource}} or {{ReadableStream}} of bytes representing an encoded
    image file as described by {{ImageDecoderInit/type}}.

: <dfn dict-member for=ImageDecoderInit>colorSpaceConversion</dfn>
:: Controls whether decoded outputs' color space is converted or ignored, as
    defined by {{ImageBitmapOptions/colorSpaceConversion}} in
    {{ImageBitmapOptions}}.

: <dfn dict-member for=ImageDecoderInit>desiredWidth</dfn>
:: Indicates a desired width for decoded outputs. Implementation is best
    effort; decoding to a desired width <em class="rfc2119">MAY</em> not be
    supported by all formats/ decoders.

: <dfn dict-member for=ImageDecoderInit>desiredHeight</dfn>
:: Indicates a desired height for decoded outputs. Implementation is best
    effort; decoding to a desired height <em class="rfc2119">MAY</em> not be
    supported by all formats/decoders.

: <dfn dict-member for=ImageDecoderInit>preferAnimation</dfn>
:: For images with multiple tracks, this indicates whether the
    initial track selection <em class="rfc2119">SHOULD</em> prefer an animated
    track.

    NOTE: See the [=ImageDecoder/Get Default Selected Track Index=] algorithm.

ImageDecodeOptions Interface {#imagedecodeoptions-interface}
------------------------------------------------------------
<pre class='idl'>
<xmp>
dictionary ImageDecodeOptions {
  [EnforceRange] unsigned long frameIndex = 0;
  boolean completeFramesOnly = true;
};
</xmp>
</pre>

: <dfn dict-member for=ImageDecodeOptions>frameIndex</dfn>
:: The index of the frame to decode.

: <dfn dict-member for=ImageDecodeOptions>completeFramesOnly</dfn>
:: For [=Progressive Images=], a value of `false` indicates that the decoder
    <em class="rfc2119">MAY</em> output an {{ImageDecodeResult/image}} with
    reduced detail. Each subsequent call to {{ImageDecoder/decode()}} for the
    same {{ImageDecodeOptions/frameIndex}} will resolve to produce an image with
    a higher [=Progressive Image Frame Generation=] (more image detail) than the
    previous call, until finally the full-detail image is produced.

    If {{ImageDecodeOptions/completeFramesOnly}} is assigned `true`, or if the
    image is not a [=Progressive Image=], or if the User Agent does not support
    progressive decoding for the given image type, calls to
    {{ImageDecoder/decode()}} will only resolve once the full detail image is
    decoded.

    <div class='note'>
      NOTE: For [=Progressive Images=], setting
          {{ImageDecodeOptions/completeFramesOnly}} to `false` can be used to
          offer users a preview an image that is still being buffered from the
          network (via the {{ImageDecoderInit/data}} {{ReadableStream}}).

          Upon decoding the full detail image, the {{ImageDecodeResult}}'s
          {{ImageDecodeResult/complete}} will be set to true.
    </div>


ImageDecodeResult Interface {#imagedecoderesult-interface}
----------------------------------------------------------
<pre class='idl'>
<xmp>
dictionary ImageDecodeResult {
  required VideoFrame image;
  required boolean complete;
};
</xmp>
</pre>

: <dfn dict-member for=ImageDecodeResult>image</dfn>
:: The decoded image.

: <dfn dict-member for=ImageDecodeResult>complete</dfn>
:: Indicates whether {{ImageDecodeResult/image}} contains the final full-detail
    output.

    NOTE: {{ImageDecodeResult/complete}} is always `true` when
        {{ImageDecoder/decode()}} is invoked with
        {{ImageDecodeOptions/completeFramesOnly}} set to `true`.

ImageTrackList Interface {#imagetracklist-interface}
----------------------------------------------------
<pre class='idl'>
<xmp>
[Exposed=(Window,DedicatedWorker), SecureContext]
interface ImageTrackList {
  getter ImageTrack (unsigned long index);

  readonly attribute Promise<undefined> ready;
  readonly attribute unsigned long length;
  readonly attribute long selectedIndex;
  readonly attribute ImageTrack? selectedTrack;
};
</xmp>
</pre>

### Internal Slots ### {#imagetracklist-internal-slots}
: <dfn attribute for=ImageTrackList>[[ready promise]]</dfn>
:: The promise used to signal when the {{ImageTrackList}} has been populated
    with {{ImageTrack}}s.

    NOTE: {{ImageTrack}} {{ImageTrack/frameCount}} can receive subsequent
        updates until {{ImageDecoder/complete}} is `true`.

: <dfn attribute for=ImageTrackList>[[track list]]</dfn>
:: The list of {{ImageTrack}}s describe by this {{ImageTrackList}}.

: <dfn attribute for=ImageTrackList>\[[selected index]]</dfn>
:: The index of the selected track in {{ImageTrackList/[[track list]]}}. A
    value of `-1` indicates that no track is selected. The initial value
    is `-1`.

### Attributes ### {#imagetracklist-attributes}
: <dfn attribute for=ImageTrackList>ready</dfn>
:: The {{ImageTrackList/ready}} getter steps are to return the
    {{ImageTrackList/[[ready promise]]}}.

: <dfn attribute for=ImageTrackList>length</dfn>
:: The {{ImageTrackList/length}} getter steps are to return the length of
    {{ImageTrackList/[[track list]]}}.

: <dfn attribute for=ImageTrackList>selectedIndex</dfn>
:: The {{ImageTrackList/selectedIndex}} getter steps are to return
    {{ImageTrackList/[[selected index]]}};

: <dfn attribute for=ImageTrackList>selectedTrack</dfn>
:: The {{ImageTrackList/selectedTrack}} getter steps are:
    1. If {{ImageTrackList/[[selected index]]}} is `-1`, return `null`.
    2. Otherwise, return the ImageTrack from {{ImageTrackList/[[track list]]}}
        at the position indicated by {{ImageTrackList/[[selected index]]}}.

ImageTrack Interface {#imagetrack-interface}
--------------------------------------------
<pre class='idl'>
<xmp>
[Exposed=(Window,DedicatedWorker), SecureContext]
interface ImageTrack {
  readonly attribute boolean animated;
  readonly attribute unsigned long frameCount;
  readonly attribute unrestricted float repetitionCount;
  attribute boolean selected;
};
</xmp>
</pre>

### Internal Slots ### {#imagetrack-internal-slots}
: <dfn attribute for=ImageTrack>\[[ImageDecoder]]</dfn>
:: The {{ImageDecoder}} instance that constructed this {{ImageTrack}}.

: <dfn attribute for=ImageTrack>\[[ImageTrackList]]</dfn>
:: The {{ImageTrackList}} instance that lists this {{ImageTrack}}.

: <dfn attribute for=ImageTrack>\[[animated]]</dfn>
:: Indicates whether this track contains an animated image with multiple
    frames.

: <dfn attribute for=ImageTrack>[[frame count]]</dfn>
:: The number of frames in this track.

: <dfn attribute for=ImageTrack>[[repetition count]]</dfn>
:: The number of times the animation is intended to repeat.

: <dfn attribute for=ImageTrack>\[[selected]]</dfn>
:: Indicates whether this track is selected for decoding.

### Attributes ### {#imagetrack-attributes}

: <dfn attribute for=ImageTrack>animated</dfn>
:: The {{ImageTrack/animated}} getter steps are to return the value of
    {{ImageTrack/[[animated]]}}.

    NOTE: This attribute provides an early indication that
        {{ImageTrack/frameCount}} will ultimately exceed 0 for images where the
        {{ImageTrack/frameCount}} starts at `0` and later increments as new
        chunks of the {{ReadableStream}} {{ImageDecoderInit/data}} arrive.

: <dfn attribute for=ImageTrack>frameCount</dfn>
:: The {{ImageTrack/frameCount}} getter steps are to return the value of
    {{ImageTrack/[[frame count]]}}.

: <dfn attribute for=ImageTrack>repetitionCount</dfn>
:: The {{ImageTrack/repetitionCount}} getter steps are to return the value of
    {{ImageTrack/[[repetition count]]}}.

: <dfn attribute for=ImageTrack>selected</dfn>
:: The {{ImageTrack/selected}} getter steps are to return the value of
    {{ImageTrack/[[selected]]}}.

    The {{ImageTrack/selected}} setter steps are:
    1. If {{ImageTrack/[[ImageDecoder]]}}'s {{ImageDecoder/[[closed]]}} slot is
        `true`, abort these steps.
    2. Let |newValue| be [=the given value=].
    3. If |newValue| equals {{ImageTrack/[[selected]]}}, abort these steps.
    4. Assign |newValue| to {{ImageTrack/[[selected]]}}.
    5. Let |parentTrackList| be {{ImageTrack/[[ImageTrackList]]}}
    6. Let |oldSelectedIndex| be the value of |parentTrackList|
        {{ImageTrackList/[[selected index]]}}.
    7. If |oldSelectedIndex| is not `-1`:
        1. Let |oldSelectedTrack| be the {{ImageTrack}} in |parentTrackList|
            {{ImageTrackList/[[track list]]}} at the position of
            |oldSelectedIndex|.
        2. Assign `false` to |oldSelectedTrack| {{ImageTrack/[[selected]]}}

    8. If |newValue| is `true`, let |selectedIndex| be the index of [=this=]
        {{ImageTrack}} within |parentTrackList|'s
        {{ImageTrackList/[[track list]]}}. Otherwise, let |selectedIndex| be
        `-1`.
    9. Assign |selectedIndex| to |parentTrackList|
        {{ImageTrackList/[[selected index]]}}.
    10. Run the [=ImageDecoder/Reset ImageDecoder=] algorithm on
        {{ImageTrack/[[ImageDecoder]]}}.
    11. [=Queue a control message=] to {{ImageTrack/[[ImageDecoder]]}}'s
        [=control message queue=] to update the internal selected track
        index with |selectedIndex|.
    12. [=Process the control message queue=] belonging to
        {{ImageTrack/[[ImageDecoder]]}}.

    [=Running a control message=] to update the internal selected track index
    means running these steps:
    1. Enqueue the following steps to {{ImageTrack/[[ImageDecoder]]}}'s
        {{ImageDecoder/[[codec work queue]]}}:
        1. Assign |selectedIndex| to
            {{ImageDecoder/[[internal selected track index]]}}.
        2. Remove all entries from
            {{ImageDecoder/[[progressive frame generations]]}}.

Resource Reclamation{#resource-reclamation}
==============================================

When resources are constrained, a User Agent <em class="rfc2119">MAY</em>
proactively reclaim codecs. This is particularly true in the case where hardware
codecs are limited, and shared accross web pages or platform apps.

To <dfn>reclaim a codec</dfn>, a User Agent <em class="rfc2119">MUST</em> run
the appropriate close algorithm (amongst [=Close AudioDecoder=],
[=Close AudioEncoder=], [=Close VideoDecoder=] and [=Close VideoEncoder=]) with
a {{QuotaExceededError}}.

The rules governing when a codec may be reclaimed depend on whether the codec is
an [=active=] or [=inactive=] codec and/or a [=background=] codec.

An <dfn lt="active codec|active">active codec</dfn> is a codec that has
made progress on the [=[[codec work queue]]=] in the past `10 seconds`.

NOTE: A reliable sign of the working queue's progress is a call to `output()`
    callback.

An <dfn lt="inactive codec|inactive">inactive codec</dfn> is any codec that does
not meet the definition of an [=active codec=].

A <dfn lt="background codec|background">background codec</dfn> is a codec whose
{{ownerDocument}} (or [=owner set=]'s {{Document}}, for codecs in workers) has a
{{Document/hidden}} attribute equal to `true`.

A User Agent <em class="rfc2119">MUST</em> only [=reclaim a codec=] that is
either an [=inactive codec=], a [=background codec=], or both. A User Agent
<em class="rfc2119">MUST NOT</em> reclaim a codec that is both [=active=] and in
the foreground, i.e. not a [=background codec=].

Additionally, User Agents <em class="rfc2119">MUST NOT</em> reclaim an [=active=]
[=background=] codec if it is:
    - An encoder, e.g. an {{AudioEncoder}} or {{VideoEncoder}}.

        NOTE: This prevents long running encode tasks from being interrupted.

    - An {{AudioDecoder}} or {{VideoDecoder}}, when there is, respectively, an
        [=active=] {{AudioEncoder}} or {{VideoEncoder}} in the same
        [=/global object=].

        NOTE: This prevents breaking long running transcoding tasks.

    - An {{AudioDecoder}}, when its tab is audibly playing audio.

Security Considerations{#security-considerations}
=================================================

<div class=non-normative>
This section is non-normative.

The primary security impact is that features of this API make it easier for an
attacker to exploit vulnerabilities in the underlying platform codecs.
Additionally, new abilities to configure and control the codecs can allow for
new exploits that rely on a specific configuration and/or sequence of control
operations.

Platform codecs are historically an internal detail of APIs like
{{HTMLMediaElement}}, [[WEBAUDIO]], and [[WebRTC]]. In this way, it has always
been possible to attack the underlying codecs by using malformed media
files/streams and invoking the various API control methods.

For example, you can send any stream to a decoder by first wrapping that stream
in a media container (e.g. mp4) and setting that as the {{HTMLMediaElement/src}}
of an {{HTMLMediaElement}}. You can then cause the underlying video decoder to
be {{VideoDecoder/reset()}} by setting a new value for `<video>.currentTime`.

WebCodecs makes such attacks easier by exposing low level control when inputs
are provided and direct access to invoke the codec control methods. This also
affords attackers the ability to invoke sequences of control methods that were
not previously possible via the higher level APIs.

The Working Group expects User Agents to mitigate this risk by extensively
fuzzing their implementation with random inputs and control method invocations.
Additionally, User Agents are encouraged to isolate their underlying codecs in
processes with restricted privileges (sandbox) as a barrier against successful
exploits being able to read user data.

An additional concern is exposing the underlying codecs to input mutation race
conditions, such as allowing a site to mutate a codec input or output while
the underlying codec is still operating on that data. This concern is mitigated
by ensuring that input and output interfaces are immutable.
</div>

Privacy Considerations{#privacy-considerations}
===============================================

<div class=non-normative>
This section is non-normative.

The primary privacy impact is an increased ability to fingerprint users by
querying for different codec capabilities to establish a codec feature profile.
Much of this profile is already exposed by existing APIs. Such profiles are very
unlikely to be uniquely identifying, but can be used with other metrics to
create a fingerprint.

An attacker can accumulate a codec feature profile by calling
`IsConfigSupported()` methods with a number of different configuration
dictionaries. Similarly, an attacker can attempt to `configure()` a codec with
different configuration dictionaries and observe which configurations are
accepted.

Attackers can also use existing APIs to establish much of the codec feature
profile. For example, the [[media-capabilities]] {{decodingInfo()}} API
describes what types of decoders are supported and its {{powerEfficient}}
attribute can signal when a decoder uses hardware acceleration. Similarly, the
[[WebRTC]] {{RTCRtpSender/getCapabilities()}} API can be used to determine what
types of encoders are supported and the {{RTCPeerConnection/getStats()}} API can
be used to determine when an encoder uses hardware acceleration. WebCodecs will
expose some additional information in the form of low level codec features.

A codec feature profile alone is unlikely to be uniquely identifying. Underlying
codecs are often implemented entirely in software (be it part of the User Agent
binary or part of the operating system), such that all users who run that
software will have a common set of capabilities. Additionally, underlying codecs
are often implemented with hardware acceleration, but such hardware is mass
produced and devices of a particular class and manufacture date (e.g. flagship
phones manufactured in 2020) will often have common capabilities. There will be
outliers (some users can be running outdated versions of software codecs or use
a rare mix of custom assembled hardware), but most of the time a given codec
feature profile is shared by a large group of users.

Segmenting groups of users by codec feature profile still amounts to a bit of
entropy that can be combined with other metrics to uniquely identify a user.
User Agents <em class="rfc2119">MAY</em> partially mitigate this by returning an
error whenever a site attempts to exhaustively probe for codec capabilities.
Additionally, User Agents <em class="rfc2119">MAY</em> implement a "privacy
budget", which depletes as authors use WebCodecs and other identifying APIs.
Upon exhaustion of the privacy budget, codec capabilities could be reduced to a
common baseline or prompt for user approval.
</div>

Best Practices for Authors Using WebCodecs{#best-practices-developers}
======================================================================

<div class=non-normative>
  This section is non-normative.

  While WebCodecs internally operates on background threads, authors working with
  realtime media or in contended main thread environments are encouraged to ensure their
  media pipelines operate in worker contexts entirely independent of the main
  thread where possible. For example, realtime media processing of {{VideoFrame}}s
  are generally to be done in a worker context.

  The main thread has significant potential for high contention and jank that can
  go unnoticed in development, yet degrade inconsistently across devices and User
  Agents in the field -- potentially dramatically impacting the end user
  experience. Ensuring the media pipeline is decoupled from the main thread helps
  provide a smooth experience for end users.

  Authors using the main thread for their media pipeline ought to be sure of
  their target frame rates, main thread workload, how their application will be
  embedded, and the class of devices their users will be using.
</div>

Acknowledgements{#acknowledgements}
===================================

The editors would like to thank Alex Russell, Chris Needham, Dale Curtis, Dan
Sanders, Eugene Zemtsov, Francois Daoust, Guido Urdaneta, Harald Alvestrand,
Jan-Ivar Bruaroey, Jer Noble, Mark Foltz, Peter Thatcher, Steve Anton, Matt
Wolenetz, Rijubrata Bhaumik, Thomas Guilbert, Tuukka Toivonen, and Youenn Fablet
for their contributions to this specification. Thank you also to the many
others who contributed to the specification, including through their
participation on the mailing list and in the issues.

The Working Group dedicates this specification to our colleague Bernard Aboba.