Adding more detailed splicing text and fixing various other bugs.
Bug 19676, 20327 - Provide more detail for audio & video splicing.
Bug 20900 - Remove complete access unit constraint.
Bug 20948 - Setting timestampOffset in 'ended' triggers a transition to 'open'
Bug 20952 - Added update event
Bug 20953 - Move end of append event firing out of segment parser loop.
Bug 21034 - Add steps to fire addtrack and removetrack events.
Binary file media-source/audio_splice.png has changed
--- a/media-source/media-source-respec.html Wed Feb 13 17:04:34 2013 -0500
+++ b/media-source/media-source-respec.html Mon Feb 18 13:44:34 2013 -0800
@@ -327,9 +327,24 @@
</li>
<li>Set the the sourceBuffer attribute in all tracks in <var>sourceBuffer</var>.<a def-id="sourcebuffer-audioTracks"></a>, <var>sourceBuffer</var>.<a def-id="sourcebuffer-videoTracks"></a>, and <var>sourceBuffer</var>.<a def-id="sourcebuffer-textTracks"></a> to null.</li>
<li>Remove all the tracks in <var>sourceBuffer</var>.<a def-id="sourcebuffer-audioTracks"></a>, <var>sourceBuffer</var>.<a def-id="sourcebuffer-videoTracks"></a>, and <var>sourceBuffer</var>.<a def-id="sourcebuffer-textTracks"></a> from the respective <a def-id="audiotracks"></a>, <a def-id="videotracks"></a>, and <a def-id="texttracks"></a> attributes on the HTMLMediaElement.</li>
- <li>Remove all the tracks in <var>sourceBuffer</var>.<a def-id="sourcebuffer-audioTracks"></a>, <var>sourceBuffer</var>.<a def-id="sourcebuffer-videoTracks"></a>, and <var>sourceBuffer</var>.<a def-id="sourcebuffer-textTracks"></a> and <a def-id="queue-a-task-to-fire-an-event-named"></a> <a def-id="tracklist-change"></a> at the modified lists.</li>
- <li><a def-id="Queue-a-task-to-fire-an-event-named"></a> <a def-id="tracklist-change"></a> at the HTMLMediaElement track lists that were modified.</li>
- <li>If <var>sourceBuffer</var> is in <a def-id="activeSourceBuffers"></a>, then remove it from <a def-id="activeSourceBuffers"></a> and <a def-id="queue-a-task-to-fire-an-event-named"></a> <a def-id="removesourcebuffer"></a> at <a def-id="activeSourceBuffers"></a>.</li>
+ <li>Remove all the tracks in <var>sourceBuffer</var>.<a def-id="sourcebuffer-audioTracks"></a>, <var>sourceBuffer</var>.<a def-id="sourcebuffer-videoTracks"></a>, and <var>sourceBuffer</var>.<a def-id="sourcebuffer-textTracks"></a> and <a def-id="queue-a-task-to-fire-an-event-named"></a> <a def-id="tracklist-removetrack"></a> at the modified lists.</li>
+ <li><a def-id="Queue-a-task-to-fire-an-event-named"></a> <a def-id="tracklist-removetrack"></a> at the HTMLMediaElement track lists that were modified.</li>
+ <li>If <var>sourceBuffer</var> is in <a def-id="activeSourceBuffers"></a>, then run the following steps:
+ <ol>
+ <li>Remove <var>sourceBuffer</var> from <a def-id="activeSourceBuffers"></a>.</li>
+ <li><a def-id="Queue-a-task-to-fire-an-event-named"></a> <a def-id="removesourcebuffer"></a> at <a def-id="activeSourceBuffers"></a>.</li>
+ <li>If the <a def-id="videoref" name="dom-videotrack-selected">selected video track</a> was removed from the <a def-id="videotracks"></a>
+ attribute on the HTMLMediaElement in a step above, then <a def-id="queue-a-task-to-fire-an-event-named"></a>
+ <a def-id="tracklist-change"></a> at the <a def-id="videotracks"></a> attribute.</li>
+ <li>If an <a def-id="videoref" name="dom-audiotrack-enabled">enabled audio track</a> was removed from the <a def-id="audiotracks"></a>
+ attribute on the HTMLMediaElement in a step above, then <a def-id="queue-a-task-to-fire-an-event-named"></a>
+ <a def-id="tracklist-change"></a> at the <a def-id="audiotracks"></a> attribute.</li>
+ <li>If a <a def-id="text-track"></a> with its <a def-id="texttrack-mode"></a> attribute set to
+ <a def-id="videoref" name="dom-texttrack-showing">"showing"</a> or <a def-id="videoref" name="dom-texttrack-hidden">"hidden"</a>
+ was removed from the <a def-id="texttracks"></a> attribute on the HTMLMediaElement in a step above, then
+ <a def-id="queue-a-task-to-fire-an-event-named"></a> <a def-id="tracklist-change"></a> at the <a def-id="texttracks"></a> attribute.</li>
+ </ol>
+ </li>
<li>Remove <var>sourceBuffer</var> from <a def-id="sourceBuffers"></a> and <a def-id="queue-a-task-to-fire-an-event-named"></a> <a def-id="removesourcebuffer"></a> at <a def-id="sourceBuffers"></a>.</li>
<li>Destroy all resources for <var>sourceBuffer</var>.</li>
</ol>
@@ -680,11 +695,17 @@
<ol>
<li>If this object has been removed from the <a def-id="sourceBuffers"></a> attribute of the <a def-id="parent-media-source"></a>, then throw an <a def-id="invalid-state-err"></a> exception and abort these steps.</li>
<li>If the <a def-id="updating"></a> attribute equals true, then throw an <a def-id="invalid-state-err"></a> exception and abort these steps.</li>
- <li>If the <a def-id="readyState"></a> attribute of the <a def-id="parent-media-source"></a> is not in the <a def-id="open"></a> state, then throw an <a def-id="invalid-state-err"></a> exception and abort these steps.</li>
+ <li>If the <a def-id="readyState"></a> attribute of the <a def-id="parent-media-source"></a> is in the <a def-id="closed"></a> state then throw an <a def-id="invalid-state-err"></a> exception and abort these steps.</li>
+ <li>
+ <p>If the <a def-id="readyState"></a> attribute of the <a def-id="parent-media-source"></a> is in the <a def-id="ended"></a> state then run the following steps:</p>
+ <ol>
+ <li>Set the <a def-id="readyState"></a> attribute of the <a def-id="parent-media-source"></a> to <a def-id="open"></a></li>
+ <li><a def-id="Queue-a-task-to-fire-an-event-named"></a> <a def-id="sourceopen"></a> at the <a def-id="parent-media-source"></a>.</li>
+ </ol>
+ </li>
<li>If this object is waiting for the end of a <a def-id="media-segment"></a> to be appended, then throw an <a def-id="invalid-state-err"></a> and abort these steps.</li>
<li>Update the attribute to the new value.</li>
</ol>
- <p class="issue"><a href="https://www.w3.org/Bugs/Public/show_bug.cgi?id=19676">Bug 19676</a> - timestampOffset accuracy</p>
</dd>
<dt>readonly attribute AudioTrackList audioTracks</dt>
@@ -728,6 +749,10 @@
<li>Set the <a def-id="updating"></a> attribute to true.</li>
<li><a def-id="Queue-a-task-to-fire-an-event-named"></a> <a def-id="updatestart"></a> at this <a>SourceBuffer</a> object.</li>
<li>Asynchronously run the <a def-id="segment-parser-loop"></a> algorithm.</li>
+ <li>When the <a def-id="segment-parser-loop"></a> returns control to this algorithm, run the remaining steps.</li>
+ <li>Set the <a def-id="updating"></a> attribute to false.</li>
+ <li><a def-id="Queue-a-task-to-fire-an-event-named"></a> <a def-id="update"></a> at this <a>SourceBuffer</a> object.</li>
+ <li><a def-id="Queue-a-task-to-fire-an-event-named"></a> <a def-id="updateend"></a> at this <a>SourceBuffer</a> object.</li>
</ol>
</dd>
@@ -811,6 +836,7 @@
<li>If <a def-id="buffer-full-flag"></a> equals true and this object is ready to accept more bytes, then set
the <a def-id="buffer-full-flag"></a> to false.</li>
<li>Set the <a def-id="updating"></a> attribute to false.</li>
+ <li><a def-id="Queue-a-task-to-fire-an-event-named"></a> <a def-id="update"></a> at this <a>SourceBuffer</a> object.</li>
<li><a def-id="Queue-a-task-to-fire-an-event-named"></a> <a def-id="updateend"></a> at this <a>SourceBuffer</a> object.</li>
</ol>
@@ -849,19 +875,24 @@
<td><a def-id="updating"></a> transitions from false to true.</td>
</tr>
<tr>
+ <td><a def-id="eventdfn">update</a></td>
+ <td><code>Event</code></td>
+ <td>The append or remove has successfully completed. <a def-id="updating"></a> transitions from true to false.</td>
+ </tr>
+ <tr>
<td><a def-id="eventdfn">updateend</a></td>
<td><code>Event</code></td>
- <td><a def-id="updating"></a> transitions from true to false.</td>
+ <td>The append or remove has ended.</td>
</tr>
<tr>
<td><a def-id="eventdfn">error</a></td>
<td><code>Event</code></td>
- <td>An error occurred during the append.</td>
+ <td>An error occurred during the append. <a def-id="updating"></a> transitions from true to false.</td>
</tr>
<tr>
<td><a def-id="eventdfn">abort</a></td>
<td><code>Event</code></td>
- <td>The append or remove was aborted by an <a def-id="abort"></a> call.</td>
+ <td>The append or remove was aborted by an <a def-id="abort"></a> call. <a def-id="updating"></a> transitions from true to false.</td>
</tr>
</tbody>
</table>
@@ -944,9 +975,7 @@
<li>Jump to the <i>loop top</i> step above.</li>
</ol>
</li>
- <li><i>Need more data:</i> If the <a def-id="stream-append-loop"></a> algorithm is running and still has data to read, then abort these steps.
- <li>Set the <a def-id="updating"></a> attribute to false.</li>
- <li><a def-id="Queue-a-task-to-fire-an-event-named"></a> <a def-id="updateend"></a> at this <a>SourceBuffer</a> object.</li>
+ <li><i>Need more data:</i> Return control to the calling algorithm.</li>
</ol>
</section>
@@ -1009,6 +1038,7 @@
<li>Run the <a def-id="segment-parser-loop"></a> algorithm.</li>
<li>Jump to the <i>loop top</i> step above.</li>
<li><i>Loop Done: </i>Set the <a def-id="updating"></a> attribute to false.</li>
+ <li><a def-id="Queue-a-task-to-fire-an-event-named"></a> <a def-id="update"></a> at this <a>SourceBuffer</a> object.</li>
<li><a def-id="Queue-a-task-to-fire-an-event-named"></a> <a def-id="updateend"></a> at this <a>SourceBuffer</a> object.</li>
</ol>
</section>
@@ -1061,7 +1091,12 @@
</ol>
</li>
<li>Add <var>new audio track</var> to the <a def-id="sourcebuffer-audioTracks"></a> attribute on this <a>SourceBuffer</a> object.</li>
+ <li><a def-id="Queue-a-task-to-fire-an-event-named"></a> <a def-id="tracklist-addtrack"></a> at
+ <a def-id="sourcebuffer-audioTracks"></a> attribute
+ on this <a>SourceBuffer</a> object.</li>
<li>Add <var>new audio track</var> to the <a def-id="audiotracks"></a> attribute on the HTMLMediaElement.</li>
+ <li><a def-id="Queue-a-task-to-fire-an-event-named"></a> <a def-id="tracklist-addtrack"></a> at the <a def-id="audiotracks"></a>
+ attribute on the HTMLMediaElement.</li>
<li>Create a new <a def-id="track-buffer"></a> to store <a def-id="coded-frames"></a> for this track.</li>
<li>Add the <a def-id="track-description"></a> for this track to the <a def-id="track-buffer"></a>.</li>
</ol>
@@ -1082,7 +1117,11 @@
</ol>
</li>
<li>Add <var>new video track</var> to the <a def-id="sourcebuffer-videoTracks"></a> attribute on this <a>SourceBuffer</a> object.</li>
+ <li><a def-id="Queue-a-task-to-fire-an-event-named"></a> <a def-id="tracklist-addtrack"></a> at <a def-id="sourcebuffer-videoTracks"></a> attribute
+ on this <a>SourceBuffer</a> object.</li>
<li>Add <var>new video track</var> to the <a def-id="videotracks"></a> attribute on the HTMLMediaElement.</li>
+ <li><a def-id="Queue-a-task-to-fire-an-event-named"></a> <a def-id="tracklist-addtrack"></a> at the <a def-id="videotracks"></a> attribute on the
+ HTMLMediaElement.</li>
<li>Create a new <a def-id="track-buffer"></a> to store <a def-id="coded-frames"></a> for this track.</li>
<li>Add the <a def-id="track-description"></a> for this track to the <a def-id="track-buffer"></a>.</li>
</ol>
@@ -1098,7 +1137,11 @@
<a def-id="texttrack-hidden"></a>, then set <var>active track flag</var> to true.
</li>
<li>Add <var>new text track</var> to the <a def-id="sourcebuffer-textTracks"></a> attribute on this <a>SourceBuffer</a> object.</li>
+ <li><a def-id="Queue-a-task-to-fire-an-event-named"></a> <a def-id="tracklist-addtrack"></a> at <a def-id="sourcebuffer-textTracks"></a> attribute
+ on this <a>SourceBuffer</a> object.</li>
<li>Add <var>new text track</var> to the <a def-id="texttracks"></a> attribute on the HTMLMediaElement.</li>
+ <li><a def-id="Queue-a-task-to-fire-an-event-named"></a> <a def-id="tracklist-addtrack"></a> at the <a def-id="texttracks"></a> attribute on the
+ HTMLMediaElement.</li>
<li>Create a new <a def-id="track-buffer"></a> to store <a def-id="coded-frames"></a> for this track.</li>
<li>Add the <a def-id="track-description"></a> for this track to the <a def-id="track-buffer"></a>.</li>
</ol>
@@ -1136,8 +1179,8 @@
<li>
<p>For each <a def-id="coded-frame"></a> in the <a def-id="media-segment"></a> run the following steps:</p>
<ol>
- <li>Let <var>presentation timestamp</var> be a double precision floating point representation of the coded frame's presentation timestamp.</li>
- <li>Let <var>decode timestamp</var> be a double precision floating point representation of the coded frame's decode timestamp.
+ <li>Let <var>presentation timestamp</var> be a double precision floating point representation of the coded frame's presentation timestamp in seconds.</li>
+ <li>Let <var>decode timestamp</var> be a double precision floating point representation of the coded frame's decode timestamp in seconds.
<p class="note">Implementations don't have to internally store timestamps in a double precision floating point representation. This
representation is used here because it is the represention for timestamps in the HTML spec. The intention here is to make the
behavior clear without adding unnecessary complexity to the algorithm to deal with the fact that adding a timestampOffset may
@@ -1146,6 +1189,7 @@
if a double precision floating point representation was used.
</p>
</li>
+ <li>Let <var>frame duration</var> be a double precision floating point representation of the coded frame's duration in seconds.</li>
<li>
<p>If <a def-id="timestampOffset"></a> is not 0, then run the following steps:</p>
<ol>
@@ -1162,25 +1206,52 @@
<a def-id="abort"></a>.</p>
</li>
- <li>If <a def-id="highest-presentation-timestamp"></a> for <var>track buffer</var> is set and less than <var>presentation timestamp</var>,
- then run the following steps:
+ <li>Let <var>frame end timestamp</var> equal the sum of <var>presentation timestamp</var> and <var>frame duration</var>.</li>
+ <li>Let <var>spliced frame</var> be an unset variable for holding audio splice information</li>
+ <li>If <a def-id="last-decode-timestamp"></a> for <var>track buffer</var> is unset and <var>presentation timestamp</var> lies within a
+ <a def-id="coded-frame"></a> already stored in <var>track buffer</var>, then run the following steps:
<ol>
- <li>Remove all <a def-id="coded-frames"></a> from <var>track buffer</var> that have a presentation timestamp greater than
- <a def-id="highest-presentation-timestamp"></a> and less than or equal to <var>presentation timestamp</var>.</li>
- <li>Remove all <a def-id="coded-frames"></a> from <var>track buffer</var> that have decoding dependencies on the coded frames removed in
- the previous step.
- <p class="note">For example if an I-frame is removed in the previous step, then all P-frames & B-frames that depend on that I-frame
- should be removed from <var>track buffer</var>. This makes sure that decode dependencies are properly maintained during overlaps.</p>
- </li>
+ <li>Let <var>overlapped frame</var> be the <a def-id="coded-frame"></a> in <var>track buffer</var> that contains <var>presentation timestamp</var>.</li>
+ <li>If <var>track buffer</var> contains audio <a def-id="coded-frames"></a>, then run the <a def-id="audio-splice-frame-algorithm"></a> and if a splice
+ frame is returned, assign it to <var>spliced frame</var>.</li>
+ <li>If <var>track buffer</var> contains video <a def-id="coded-frames"></a> and <var>presentation timestamp</var> is less than 1 microsecond
+ beyond the presentation timestamp of <var>overlapped frame</var>, then remove <var>overlapped frame</var> and any
+ <a def-id="coded-frames"></a> that depend on it from <var>track buffer</var>.
+ <p class="note">
+ This is to compensate for minor errors in frame timestamp computations that can appear when converting back and forth between double precision
+ floating point numbers and rationals. This tolerance allows a frame to replace an existing one as long as it is within 1 microsecond of the existing
+ frame's start time. Frames that come slightly before an existing frame are handled by the removal step below.
+ </p>
</li>
</ol>
</li>
- <li>Add the <a def-id="coded-frame"></a> with the <var>presentation timestamp</var> and <var>decode timestamp</var>, to the
- <var>track buffer</var>.</li>
+ <li>Remove existing coded frames in <var>track buffer</var>:
+ <dl class="switch">
+ <dt>If <a def-id="highest-presentation-timestamp"></a> for <var>track buffer</var> is not set:</dt>
+ <dd>Remove all <a def-id="coded-frames"></a> from <var>track buffer</var> that have a presentation timestamp greater than or equal to
+ <var>presentation timestamp</var> and less than <var>frame end timestamp</var>.</dd>
+ <dt>If <a def-id="highest-presentation-timestamp"></a> for <var>track buffer</var> is set and less than <var>presentation timestamp</var></dt>
+ <dd>Remove all <a def-id="coded-frames"></a> from <var>track buffer</var> that have a presentation timestamp greater than
+ <a def-id="highest-presentation-timestamp"></a> and less than or equal to <var>frame end timestamp</var>.</dd>
+ </dl>
+ </li>
+ <li>Remove all <a def-id="coded-frames"></a> from <var>track buffer</var> that have decoding dependencies on the coded frames removed in
+ the previous step.
+ <p class="note">For example if an I-frame is removed in the previous step, then all P-frames & B-frames that depend on that I-frame
+ should be removed from <var>track buffer</var>. This makes sure that decode dependencies are properly maintained during overlaps.</p>
+ </li>
+ <li>
+ <dl class="switch">
+ <dt>If <var>spliced frame</var> is set:</dt>
+ <dd>Add <var>spliced frame</var> to the <var>track buffer</var>.</dd>
+ <dt>Otherwise:</dt>
+ <dd>Add the <a def-id="coded-frame"></a> with the <var>presentation timestamp</var>, <var>decode timestamp</var>, and <var>frame duration</var> to the
+ <var>track buffer</var>.</dd>
+ </dl>
<li>Set <a def-id="last-decode-timestamp"></a> for <var>track buffer</var> to <var>decode timestamp</var>.</li>
- <li>If <a def-id="highest-presentation-timestamp"></a> for <var>track buffer</var> is unset or <var>presentation timestamp</var> is greater
+ <li>If <a def-id="highest-presentation-timestamp"></a> for <var>track buffer</var> is unset or <var>frame end timestamp</var> is greater
than <a def-id="highest-presentation-timestamp"></a>, then set <a def-id="highest-presentation-timestamp"></a> for <var>track buffer</var>
- to <var>presentation timestamp</var>.
+ to <var>frame end timestamp</var>.
<p class="note">The greater than check is needed because bidirectional prediction between coded frames can cause
<var>presentation timestamp</var> to not be monotonically increasing eventhough the decode timestamps are monotonically increasing.</p>
</li>
@@ -1212,6 +1283,97 @@
<li>If the <a def-id="media-segment"></a> contains data beyond the current <a def-id="duration"></a>, then run the <a def-id="duration-change-algorithm"></a> with <var>new duration</var> set to the maximum of the current duration and the highest end timestamp reported by <a def-id="hme-buffered"></a>.</li>
</ol>
</section>
+
+ <section id="sourcebuffer-audio-splice-frame-algorithm">
+ <h4>Audio Splice Frame Algorithm</h4>
+ <p>Follow these steps when the <a def-id="coded-frame-processing-algorithm"></a> needs to generate a splice frame for two overlapping audio
+ <a def-id="coded-frames"></a>:</p>
+ <ol>
+ <li>Let <var>track buffer</var> be the <a def-id="track-buffer"></a> that will contain the splice.</li>
+ <li>Let <var>new coded frame</var> be the new <a def-id="coded-frame"></a>, that is being added to <var>track buffer</var>, which triggered the need for a splice.</li>
+ <li>Let <var>presentation timestamp</var> be the presentation timestamp for <var>new coded frame</var></li>
+ <li>Let <var>decode timestamp</var> be the decode timestamp for <var>new coded frame</var>.</li>
+ <li>Let <var>frame duration</var> be the duration of <var>new coded frame</var>.</li>
+ <li>Let <var>overlapped frame</var> be the <a def-id="coded-frame"></a> in <var>track buffer</var> that overlaps with <var>new coded frame</var>
+ (ie. it contains <var>presentation timestamp).</li>
+ <li>Round & update <var>presentation timestamp</var> and <var>decode timestamp</var> to the nearest audio sample timestamp based on sample rate of the audio in <var>overlapped frame</var>.
+ <div class="note">
+ <p>For example, given the following values:</p>
+ <ul>
+ <li>The presentation timestamp of <var>overlapped frame</var> equals 10.</li>
+ <li>The sample rate of <var>overlapped frame</var> equals 8000 Hz</li>
+ <li><var>presentation timestamp</var> equals 10.01255</li>
+ <li><var>decode timestamp</var> equals 10.01255</li>
+ </ul>
+ <p><var>presentation timestamp</var> and <var>decode timestamp</var> are rounded & updated to 10.0125 since 10.01255 is closer to
+ 10 + 100/8000 (10.0125) than 10 + 101/8000 (10.012625)</p>
+ </div>
+ </li>
+ <li>If the user agent does not support crossfading then run the following steps:
+ <ol>
+ <li>Remove <var>overlapped frame</var> from <var>track buffer</var>.</li>
+ <li>Add a silence frame to <var>track buffer</var> with the following properties:
+ <ul>
+ <li>The presentation time set to the <var>overlapped frame</var> presentation time.</li>
+ <li>The decode time set to the <var>overlapped frame</var> decode time.</li>
+ <li>The frame duration set to difference between <var>presentation timestamp</var> and the <var>overlapped frame</var> presentation time.</li>
+ </ul>
+ </li>
+ <li>Return to caller without providing a splice frame.
+ <p class="note">
+ This is intended to allow <var>new coded frame</var> to be added to the <var>track buffer</var> as if
+ <var>overlapped frame</var> had not been in the <var>track buffer</var> to begin with.
+ </p>
+ </li>
+ </ol>
+ </li>
+ <li>Let <var>frame end timestamp</var> equal the sum of <var>presentation timestamp</var> and <var>frame duration</var>.</li>
+ <li>Let <var>fade out coded frames</var> equal <var>overlapped frame</var> as well as any addition frames in <var>track buffer</var> that overlap
+ <var>presentation timestamp</var> plus the splice duration of 5 milliseconds.</li>
+ <li>Remove all the frames included in <var>fade out coded frames</var> from <var>track buffer</var>.
+ <li>Return a splice frame with the following properties:
+ <ul>
+ <li>The presentation time set to the <var>overlapped frame</var> presentation time.</li>
+ <li>The decode time set to the <var>overlapped frame</var> decode time.</li>
+ <li>The frame duration set to difference between <var>frame end timestamp</var> and the <var>overlapped frame</var> presentation time.</li>
+ <li>The fade out coded frames equals <var>fade-out coded frames</var>.</li>
+ <li>The fade in coded frame equals <var>new coded frame</var>.</li>
+ <li>The splice timestamp equals <var>presentation timestamp</var>.</li>
+ </ul>
+ <p class="note">See the <a def-id="audio-splice-rendering-algorithm"></a> for details on how this splice frame is rendered.</p>
+ </li>
+ </ol>
+ </section>
+ <section id="sourcebuffer-audio-splice-rendering-algorithm">
+ <h4>Audio Splice Rendering Algorithm</h4>
+ <p>The following steps are run when a spliced frame, generated by the <a def-id="audio-splice-frame-algorithm"></a>, needs to be rendered by the
+ media element:</p>
+ <ol>
+ <li>Let <var>fade out coded frames</var> be the <a def-id="coded-frames"></a> that are faded out during the splice.</li>
+ <li>Let <var>fade in coded frames</var> be the <a def-id="coded-frames"></a> that are faded in during the splice.</li>
+ <li>Let <var>presentation timestamp</var> be the presentation timestamp of the first coded frame in <var>fade out coded frames</var>.</li>
+ <li>Let <var>end timestamp</var> be the sum of the presentation timestamp and frame duration in the last frame in <var>fade in coded frames</var>.</li>
+ <li>Let <var>splice timestamp</var> be the presentation timestamp where the splice starts. This corresponds with the presentation timestamp of the first frame in
+ <var>fade in coded frames</var>.</li>
+ <li>Let <var>splice end timestamp</var> equal <var>splice timestamp</var> plus five milliseconds.</li>
+ <li>Let <var>fade out samples</var> be the samples generated by decoding <var>fade out coded frames</var>.</li>
+ <li>Trim <var>fade out samples</var> so that it only contains samples between <var>presentation timestamp</var> and <var>splice end timestamp</var>.</li>
+ <li>Let <var>fade in samples</var> be the samples generated by decoding <var>fade in coded frames</var>.</li>
+ <li>Convert <var>fade out samples</var> and <var>fade in samples</var> to a common sample rate and channel layout.</li>
+ <li>Let <var>output samples</var> be a buffer to hold the output samples.</li>
+ <li>Apply a linear gain fade out to the samples between <var>splice timestamp</var> and <var>splice end timestamp</var> in <var>fade out samples</var>.</li>
+ <li>Apply a linear gain fade in to the samples between <var>splice timestamp</var> and <var>splice end timestamp</var> in <var>fade in samples</var>.</li>
+ <li>Copy samples between <var>presentation timestamp</var> to <var>splice timestamp</var> from <var>fade out samples</var> into <var>output samples</var>.</li>
+ <li>For each sample between <var>splice timestamp</var> and <var>splice end timestamp</var>, compute the sum of a sample from <var>fade out samples</var> and the
+ corresponding sample in <var>fade in samples</var> and store the result in <var>output samples</var>.</li>
+ <li>Copy samples between <var>splice end timestamp</var> to <var>end timestamp</var> from <var>fade in samples</var> into <var>output samples</var>.</li>
+ <li>Render <var>output samples</var>.</li>
+ </ol>
+ <div class="note">
+ <p>Here is a graphical representation of this algorithm.</p>
+ <img src="audio_splice.png" alt="Audio splice diagram">
+ </div>
+ </section>
</section>
</section>
@@ -1312,8 +1474,6 @@
</li>
<li>Return the <var>intersection ranges</var>.</li>
</ol>
- <p class="issue"><a href="https://www.w3.org/Bugs/Public/show_bug.cgi?id=18615">Bug 18615</a> - Define how SourceBuffer.buffered maps to HTMLMediaElement.buffered</p>
- <p class="issue"><a href="https://www.w3.org/Bugs/Public/show_bug.cgi?id=18400">Bug 18400</a> - Define and document timestamp heuristics</p>
</section>
<section id="audio-track-extensions">
@@ -1637,7 +1797,6 @@
<li>PSI that is identical to the information in the initialization segment may appear repeatedly throughout the segment.</li>
<li>The media segment will not rely on initialization information in another media segment.</li>
<li>Media Segments must contain only complete PES packets and sections.</li>
- <li>Each PES packet must be comprised of one or more complete access units.</li>
<li>Each PES packet must have a PTS timestamp.</li>
<li>PCR must be present in the Segment prior to the first byte of a TS packet payload containing media data.</li>
<li>The presentation duration of each media component within the Media Segment should be as close to equal as practical.</li>
@@ -1777,7 +1936,20 @@
</thead>
<tbody>
<tr>
- <td>05 February 2013</a></td>
+ <td>19 February 2013</td>
+ <td>
+ <ul>
+ <li>Bug 19676, 20327 - Provide more detail for audio & video splicing.</li>
+ <li>Bug 20900 - Remove complete access unit constraint.</li>
+ <li>Bug 20948 - Setting timestampOffset in 'ended' triggers a transition to 'open'</li>
+ <li>Bug 20952 - Added update event.</li>
+ <li>Bug 20953 - Move end of append event firing out of segment parser loop.</li>
+ <li>Bug 21034 - Add steps to fire addtrack and removetrack events.</li>
+ </ul>
+ </td>
+ </tr>
+ <tr>
+ <td><a href="https://dvcs.w3.org/hg/html-media/raw-file/77975abeec41/media-source/media-source.html">05 February 2013</a></td>
<td>
<ul>
<li>Bug 19676 - Added a note clarifying that the internal timestamp representation doesn't have to be a double.</li>
@@ -1877,11 +2049,11 @@
<td>Converted document to ReSpec.</td>
</tr>
<tr>
- <td><a href="http://dvcs.w3.org/hg/html-media/rev/e029f71aafca">18 October 2012</a></td>
+ <td><a href="http://dvcs.w3.org/hg/html-media/raw-file/e029f71aafca/media-source/media-source.html">18 October 2012</a></td>
<td>Refactored SourceBuffer.append() & added SourceBuffer.remove().</td>
</tr>
<tr>
- <td><a href="http://dvcs.w3.org/hg/html-media/rev/6d127e69c9f8">8 October 2012</a></td>
+ <td><a href="http://dvcs.w3.org/hg/html-media/raw-file/6d127e69c9f8/media-source/media-source.html">8 October 2012</a></td>
<td>
<ul>
<li>Defined what HTMLMediaElement.seekable and HTMLMediaElement.buffered should return.</li>
--- a/media-source/media-source.html Wed Feb 13 17:04:34 2013 -0500
+++ b/media-source/media-source.html Mon Feb 18 13:44:34 2013 -0800
@@ -397,7 +397,7 @@
</p>
<h1 class="title" id="title">Media Source Extensions</h1>
- <h2 id="w3c-editor-s-draft-05-february-2013"><abbr title="World Wide Web Consortium">W3C</abbr> Editor's Draft 05 February 2013</h2>
+ <h2 id="w3c-editor-s-draft-19-february-2013"><abbr title="World Wide Web Consortium">W3C</abbr> Editor's Draft 19 February 2013</h2>
<dl>
<dt>This version:</dt>
@@ -507,7 +507,7 @@
-</section><section id="toc"><h2 class="introductory">Table of Contents</h2><ul class="toc"><li class="tocline"><a href="#introduction" class="tocxref"><span class="secno">1. </span>Introduction</a><ul class="toc"><li class="tocline"><a href="#goals" class="tocxref"><span class="secno">1.1 </span>Goals</a></li><li class="tocline"><a href="#definitions" class="tocxref"><span class="secno">1.2 </span>Definitions</a></li></ul></li><li class="tocline"><a href="#source-buffer-model" class="tocxref"><span class="secno">2. </span>Source Buffer Model</a><ul class="toc"><li class="tocline"><a href="#source-buffer-overlapping-segments" class="tocxref"><span class="secno">2.1 </span>Appending a Media Segment over a buffered region</a><ul class="toc"><li class="tocline"><a href="#source-buffer-overlap-complete" class="tocxref"><span class="secno">2.1.1 </span>Complete Overlap</a></li><li class="tocline"><a href="#source-buffer-overlap-start" class="tocxref"><span class="secno">2.1.2 </span>Start Overlap</a></li><li class="tocline"><a href="#source-buffer-overlap-end" class="tocxref"><span class="secno">2.1.3 </span>End Overlap</a></li><li class="tocline"><a href="#source-buffer-overlap-middle" class="tocxref"><span class="secno">2.1.4 </span>Middle Overlap</a></li></ul></li><li class="tocline"><a href="#track-buffer-to-decoder-buffer" class="tocxref"><span class="secno">2.2 </span>Track Buffer to Decoder Buffer transfer</a></li></ul></li><li class="tocline"><a href="#mediasource" class="tocxref"><span class="secno">3. </span>MediaSource Object</a><ul class="toc"><li class="tocline"><a href="#attributes" class="tocxref"><span class="secno">3.1 </span>Attributes</a></li><li class="tocline"><a href="#methods" class="tocxref"><span class="secno">3.2 </span>Methods</a></li><li class="tocline"><a href="#mediasource-events" class="tocxref"><span class="secno">3.3 </span>Event Summary</a></li><li class="tocline"><a href="#mediasource-algorithms" class="tocxref"><span class="secno">3.4 </span>Algorithms</a><ul class="toc"><li class="tocline"><a href="#mediasource-attach" class="tocxref"><span class="secno">3.4.1 </span>Attaching to a media element</a></li><li class="tocline"><a href="#mediasource-detach" class="tocxref"><span class="secno">3.4.2 </span>Detaching from a media element</a></li><li class="tocline"><a href="#mediasource-seeking" class="tocxref"><span class="secno">3.4.3 </span>Seeking</a></li><li class="tocline"><a href="#buffer-monitoring" class="tocxref"><span class="secno">3.4.4 </span>SourceBuffer Monitoring</a></li><li class="tocline"><a href="#active-source-buffer-changes" class="tocxref"><span class="secno">3.4.5 </span>Changes to selected/enabled track state</a></li><li class="tocline"><a href="#duration-change-algorithm" class="tocxref"><span class="secno">3.4.6 </span>Duration change</a></li></ul></li></ul></li><li class="tocline"><a href="#sourcebuffer" class="tocxref"><span class="secno">4. </span>SourceBuffer Object</a><ul class="toc"><li class="tocline"><a href="#attributes-1" class="tocxref"><span class="secno">4.1 </span>Attributes</a></li><li class="tocline"><a href="#methods-1" class="tocxref"><span class="secno">4.2 </span>Methods</a></li><li class="tocline"><a href="#track-buffers" class="tocxref"><span class="secno">4.3 </span>Track Buffers</a></li><li class="tocline"><a href="#sourcebuffer-events" class="tocxref"><span class="secno">4.4 </span>Event Summary</a></li><li class="tocline"><a href="#sourcebuffer-algorithms" class="tocxref"><span class="secno">4.5 </span>Algorithms</a><ul class="toc"><li class="tocline"><a href="#sourcebuffer-segment-parser-loop" class="tocxref"><span class="secno">4.5.1 </span>Segment Parser Loop</a></li><li class="tocline"><a href="#sourcebuffer-reset-parser-state" class="tocxref"><span class="secno">4.5.2 </span>Reset Parser State</a></li><li class="tocline"><a href="#sourcebuffer-append-error" class="tocxref"><span class="secno">4.5.3 </span>Append Error</a></li><li class="tocline"><a href="#sourcebuffer-stream-append-loop" class="tocxref"><span class="secno">4.5.4 </span>Stream Append Loop</a></li><li class="tocline"><a href="#sourcebuffer-init-segment-received" class="tocxref"><span class="secno">4.5.5 </span>Initialization Segment Received</a></li><li class="tocline"><a href="#sourcebuffer-coded-frame-processing" class="tocxref"><span class="secno">4.5.6 </span>Coded Frame Processing</a></li></ul></li></ul></li><li class="tocline"><a href="#sourcebufferlist" class="tocxref"><span class="secno">5. </span>SourceBufferList Object</a><ul class="toc"><li class="tocline"><a href="#attributes-2" class="tocxref"><span class="secno">5.1 </span>Attributes</a></li><li class="tocline"><a href="#methods-2" class="tocxref"><span class="secno">5.2 </span>Methods</a></li><li class="tocline"><a href="#sourcebufferlist-events" class="tocxref"><span class="secno">5.3 </span>Event Summary</a></li></ul></li><li class="tocline"><a href="#url" class="tocxref"><span class="secno">6. </span>URL Object</a><ul class="toc"><li class="tocline"><a href="#methods-3" class="tocxref"><span class="secno">6.1 </span>Methods</a></li></ul></li><li class="tocline"><a href="#htmlmediaelement-attributes" class="tocxref"><span class="secno">7. </span>HTMLMediaElement attributes</a></li><li class="tocline"><a href="#audio-track-extensions" class="tocxref"><span class="secno">8. </span>AudioTrack Extensions</a><ul class="toc"><li class="tocline"><a href="#attributes-3" class="tocxref"><span class="secno">8.1 </span>Attributes</a></li></ul></li><li class="tocline"><a href="#video-track-extensions" class="tocxref"><span class="secno">9. </span>VideoTrack Extensions</a><ul class="toc"><li class="tocline"><a href="#attributes-4" class="tocxref"><span class="secno">9.1 </span>Attributes</a></li></ul></li><li class="tocline"><a href="#text-track-extensions" class="tocxref"><span class="secno">10. </span>TextTrack Extensions</a><ul class="toc"><li class="tocline"><a href="#attributes-5" class="tocxref"><span class="secno">10.1 </span>Attributes</a></li></ul></li><li class="tocline"><a href="#byte-stream-formats" class="tocxref"><span class="secno">11. </span>Byte Stream Formats</a><ul class="toc"><li class="tocline"><a href="#webm" class="tocxref"><span class="secno">11.1 </span>WebM Byte Streams</a><ul class="toc"><li class="tocline"><a href="#webm-init-segments" class="tocxref"><span class="secno">11.1.1 </span>Initialization Segments</a></li><li class="tocline"><a href="#webm-media-segments" class="tocxref"><span class="secno">11.1.2 </span>Media Segments</a></li><li class="tocline"><a href="#webm-random-access-points" class="tocxref"><span class="secno">11.1.3 </span>Random Access Points</a></li></ul></li><li class="tocline"><a href="#iso" class="tocxref"><span class="secno">11.2 </span>ISO Base Media File Format Byte Streams</a><ul class="toc"><li class="tocline"><a href="#iso-init-segments" class="tocxref"><span class="secno">11.2.1 </span>Initialization Segments</a></li><li class="tocline"><a href="#iso-media-segments" class="tocxref"><span class="secno">11.2.2 </span>Media Segments</a></li><li class="tocline"><a href="#iso-random-access-points" class="tocxref"><span class="secno">11.2.3 </span>Random Access Points</a></li></ul></li><li class="tocline"><a href="#mpeg2ts" class="tocxref"><span class="secno">11.3 </span>MPEG-2 Transport Stream Byte Streams</a><ul class="toc"><li class="tocline"><a href="#mpeg2ts-general" class="tocxref"><span class="secno">11.3.1 </span>General</a></li><li class="tocline"><a href="#mpeg2ts-init-segments" class="tocxref"><span class="secno">11.3.2 </span>Initialization Segments</a></li><li class="tocline"><a href="#mpeg2ts-media-segments" class="tocxref"><span class="secno">11.3.3 </span>Media Segments</a></li><li class="tocline"><a href="#mpeg2ts-random-access-points" class="tocxref"><span class="secno">11.3.4 </span>Random Access Points</a></li><li class="tocline"><a href="#mpeg2ts-discontinuities" class="tocxref"><span class="secno">11.3.5 </span>Timestamp Rollover & Discontinuities</a></li></ul></li></ul></li><li class="tocline"><a href="#examples" class="tocxref"><span class="secno">12. </span>Examples</a></li><li class="tocline"><a href="#revision-history" class="tocxref"><span class="secno">13. </span>Revision History</a></li><li class="tocline"><a href="#references" class="tocxref"><span class="secno">A. </span>References</a><ul class="toc"><li class="tocline"><a href="#informative-references" class="tocxref"><span class="secno">A.1 </span>Informative references</a></li></ul></li></ul></section>
+</section><section id="toc"><h2 class="introductory">Table of Contents</h2><ul class="toc"><li class="tocline"><a href="#introduction" class="tocxref"><span class="secno">1. </span>Introduction</a><ul class="toc"><li class="tocline"><a href="#goals" class="tocxref"><span class="secno">1.1 </span>Goals</a></li><li class="tocline"><a href="#definitions" class="tocxref"><span class="secno">1.2 </span>Definitions</a></li></ul></li><li class="tocline"><a href="#source-buffer-model" class="tocxref"><span class="secno">2. </span>Source Buffer Model</a><ul class="toc"><li class="tocline"><a href="#source-buffer-overlapping-segments" class="tocxref"><span class="secno">2.1 </span>Appending a Media Segment over a buffered region</a><ul class="toc"><li class="tocline"><a href="#source-buffer-overlap-complete" class="tocxref"><span class="secno">2.1.1 </span>Complete Overlap</a></li><li class="tocline"><a href="#source-buffer-overlap-start" class="tocxref"><span class="secno">2.1.2 </span>Start Overlap</a></li><li class="tocline"><a href="#source-buffer-overlap-end" class="tocxref"><span class="secno">2.1.3 </span>End Overlap</a></li><li class="tocline"><a href="#source-buffer-overlap-middle" class="tocxref"><span class="secno">2.1.4 </span>Middle Overlap</a></li></ul></li><li class="tocline"><a href="#track-buffer-to-decoder-buffer" class="tocxref"><span class="secno">2.2 </span>Track Buffer to Decoder Buffer transfer</a></li></ul></li><li class="tocline"><a href="#mediasource" class="tocxref"><span class="secno">3. </span>MediaSource Object</a><ul class="toc"><li class="tocline"><a href="#attributes" class="tocxref"><span class="secno">3.1 </span>Attributes</a></li><li class="tocline"><a href="#methods" class="tocxref"><span class="secno">3.2 </span>Methods</a></li><li class="tocline"><a href="#mediasource-events" class="tocxref"><span class="secno">3.3 </span>Event Summary</a></li><li class="tocline"><a href="#mediasource-algorithms" class="tocxref"><span class="secno">3.4 </span>Algorithms</a><ul class="toc"><li class="tocline"><a href="#mediasource-attach" class="tocxref"><span class="secno">3.4.1 </span>Attaching to a media element</a></li><li class="tocline"><a href="#mediasource-detach" class="tocxref"><span class="secno">3.4.2 </span>Detaching from a media element</a></li><li class="tocline"><a href="#mediasource-seeking" class="tocxref"><span class="secno">3.4.3 </span>Seeking</a></li><li class="tocline"><a href="#buffer-monitoring" class="tocxref"><span class="secno">3.4.4 </span>SourceBuffer Monitoring</a></li><li class="tocline"><a href="#active-source-buffer-changes" class="tocxref"><span class="secno">3.4.5 </span>Changes to selected/enabled track state</a></li><li class="tocline"><a href="#duration-change-algorithm" class="tocxref"><span class="secno">3.4.6 </span>Duration change</a></li></ul></li></ul></li><li class="tocline"><a href="#sourcebuffer" class="tocxref"><span class="secno">4. </span>SourceBuffer Object</a><ul class="toc"><li class="tocline"><a href="#attributes-1" class="tocxref"><span class="secno">4.1 </span>Attributes</a></li><li class="tocline"><a href="#methods-1" class="tocxref"><span class="secno">4.2 </span>Methods</a></li><li class="tocline"><a href="#track-buffers" class="tocxref"><span class="secno">4.3 </span>Track Buffers</a></li><li class="tocline"><a href="#sourcebuffer-events" class="tocxref"><span class="secno">4.4 </span>Event Summary</a></li><li class="tocline"><a href="#sourcebuffer-algorithms" class="tocxref"><span class="secno">4.5 </span>Algorithms</a><ul class="toc"><li class="tocline"><a href="#sourcebuffer-segment-parser-loop" class="tocxref"><span class="secno">4.5.1 </span>Segment Parser Loop</a></li><li class="tocline"><a href="#sourcebuffer-reset-parser-state" class="tocxref"><span class="secno">4.5.2 </span>Reset Parser State</a></li><li class="tocline"><a href="#sourcebuffer-append-error" class="tocxref"><span class="secno">4.5.3 </span>Append Error</a></li><li class="tocline"><a href="#sourcebuffer-stream-append-loop" class="tocxref"><span class="secno">4.5.4 </span>Stream Append Loop</a></li><li class="tocline"><a href="#sourcebuffer-init-segment-received" class="tocxref"><span class="secno">4.5.5 </span>Initialization Segment Received</a></li><li class="tocline"><a href="#sourcebuffer-coded-frame-processing" class="tocxref"><span class="secno">4.5.6 </span>Coded Frame Processing</a></li><li class="tocline"><a href="#sourcebuffer-audio-splice-frame-algorithm" class="tocxref"><span class="secno">4.5.7 </span>Audio Splice Frame Algorithm</a></li><li class="tocline"><a href="#sourcebuffer-audio-splice-rendering-algorithm" class="tocxref"><span class="secno">4.5.8 </span>Audio Splice Rendering Algorithm</a></li></ul></li></ul></li><li class="tocline"><a href="#sourcebufferlist" class="tocxref"><span class="secno">5. </span>SourceBufferList Object</a><ul class="toc"><li class="tocline"><a href="#attributes-2" class="tocxref"><span class="secno">5.1 </span>Attributes</a></li><li class="tocline"><a href="#methods-2" class="tocxref"><span class="secno">5.2 </span>Methods</a></li><li class="tocline"><a href="#sourcebufferlist-events" class="tocxref"><span class="secno">5.3 </span>Event Summary</a></li></ul></li><li class="tocline"><a href="#url" class="tocxref"><span class="secno">6. </span>URL Object</a><ul class="toc"><li class="tocline"><a href="#methods-3" class="tocxref"><span class="secno">6.1 </span>Methods</a></li></ul></li><li class="tocline"><a href="#htmlmediaelement-attributes" class="tocxref"><span class="secno">7. </span>HTMLMediaElement attributes</a></li><li class="tocline"><a href="#audio-track-extensions" class="tocxref"><span class="secno">8. </span>AudioTrack Extensions</a><ul class="toc"><li class="tocline"><a href="#attributes-3" class="tocxref"><span class="secno">8.1 </span>Attributes</a></li></ul></li><li class="tocline"><a href="#video-track-extensions" class="tocxref"><span class="secno">9. </span>VideoTrack Extensions</a><ul class="toc"><li class="tocline"><a href="#attributes-4" class="tocxref"><span class="secno">9.1 </span>Attributes</a></li></ul></li><li class="tocline"><a href="#text-track-extensions" class="tocxref"><span class="secno">10. </span>TextTrack Extensions</a><ul class="toc"><li class="tocline"><a href="#attributes-5" class="tocxref"><span class="secno">10.1 </span>Attributes</a></li></ul></li><li class="tocline"><a href="#byte-stream-formats" class="tocxref"><span class="secno">11. </span>Byte Stream Formats</a><ul class="toc"><li class="tocline"><a href="#webm" class="tocxref"><span class="secno">11.1 </span>WebM Byte Streams</a><ul class="toc"><li class="tocline"><a href="#webm-init-segments" class="tocxref"><span class="secno">11.1.1 </span>Initialization Segments</a></li><li class="tocline"><a href="#webm-media-segments" class="tocxref"><span class="secno">11.1.2 </span>Media Segments</a></li><li class="tocline"><a href="#webm-random-access-points" class="tocxref"><span class="secno">11.1.3 </span>Random Access Points</a></li></ul></li><li class="tocline"><a href="#iso" class="tocxref"><span class="secno">11.2 </span>ISO Base Media File Format Byte Streams</a><ul class="toc"><li class="tocline"><a href="#iso-init-segments" class="tocxref"><span class="secno">11.2.1 </span>Initialization Segments</a></li><li class="tocline"><a href="#iso-media-segments" class="tocxref"><span class="secno">11.2.2 </span>Media Segments</a></li><li class="tocline"><a href="#iso-random-access-points" class="tocxref"><span class="secno">11.2.3 </span>Random Access Points</a></li></ul></li><li class="tocline"><a href="#mpeg2ts" class="tocxref"><span class="secno">11.3 </span>MPEG-2 Transport Stream Byte Streams</a><ul class="toc"><li class="tocline"><a href="#mpeg2ts-general" class="tocxref"><span class="secno">11.3.1 </span>General</a></li><li class="tocline"><a href="#mpeg2ts-init-segments" class="tocxref"><span class="secno">11.3.2 </span>Initialization Segments</a></li><li class="tocline"><a href="#mpeg2ts-media-segments" class="tocxref"><span class="secno">11.3.3 </span>Media Segments</a></li><li class="tocline"><a href="#mpeg2ts-random-access-points" class="tocxref"><span class="secno">11.3.4 </span>Random Access Points</a></li><li class="tocline"><a href="#mpeg2ts-discontinuities" class="tocxref"><span class="secno">11.3.5 </span>Timestamp Rollover & Discontinuities</a></li></ul></li></ul></li><li class="tocline"><a href="#examples" class="tocxref"><span class="secno">12. </span>Examples</a></li><li class="tocline"><a href="#revision-history" class="tocxref"><span class="secno">13. </span>Revision History</a></li><li class="tocline"><a href="#references" class="tocxref"><span class="secno">A. </span>References</a><ul class="toc"><li class="tocline"><a href="#informative-references" class="tocxref"><span class="secno">A.1 </span>Informative references</a></li></ul></li></ul></section>
<section id="introduction">
@@ -801,9 +801,24 @@
</li>
<li>Set the the sourceBuffer attribute in all tracks in <var>sourceBuffer</var>.<code><a href="#widl-SourceBuffer-audioTracks">audioTracks</a></code>, <var>sourceBuffer</var>.<code><a href="#widl-SourceBuffer-videoTracks">videoTracks</a></code>, and <var>sourceBuffer</var>.<code><a href="#widl-SourceBuffer-textTracks">textTracks</a></code> to null.</li>
<li>Remove all the tracks in <var>sourceBuffer</var>.<code><a href="#widl-SourceBuffer-audioTracks">audioTracks</a></code>, <var>sourceBuffer</var>.<code><a href="#widl-SourceBuffer-videoTracks">videoTracks</a></code>, and <var>sourceBuffer</var>.<code><a href="#widl-SourceBuffer-textTracks">textTracks</a></code> from the respective <code><a href="http://www.w3.org/TR/html5/embedded-content-0.html#dom-media-audiotracks">audioTracks</a></code>, <code><a href="http://www.w3.org/TR/html5/embedded-content-0.html#dom-media-videotracks">videoTracks</a></code>, and <code><a href="http://www.w3.org/TR/html5/embedded-content-0.html#dom-media-texttracks">textTracks</a></code> attributes on the HTMLMediaElement.</li>
- <li>Remove all the tracks in <var>sourceBuffer</var>.<code><a href="#widl-SourceBuffer-audioTracks">audioTracks</a></code>, <var>sourceBuffer</var>.<code><a href="#widl-SourceBuffer-videoTracks">videoTracks</a></code>, and <var>sourceBuffer</var>.<code><a href="#widl-SourceBuffer-textTracks">textTracks</a></code> and <a href="http://www.w3.org/TR/html5/webappapis.html#queue-a-task">queue a task</a> to <a href="http://www.w3.org/TR/html5/webappapis.html#fire-a-simple-event">fire a simple event</a> named <code><a href="http://www.w3.org/TR/html5/embedded-content-0.html#handler-tracklist-onchange">change</a></code> at the modified lists.</li>
- <li><a href="http://www.w3.org/TR/html5/webappapis.html#queue-a-task">Queue a task</a> to <a href="http://www.w3.org/TR/html5/webappapis.html#fire-a-simple-event">fire a simple event</a> named <code><a href="http://www.w3.org/TR/html5/embedded-content-0.html#handler-tracklist-onchange">change</a></code> at the HTMLMediaElement track lists that were modified.</li>
- <li>If <var>sourceBuffer</var> is in <code><a href="#widl-MediaSource-activeSourceBuffers">activeSourceBuffers</a></code>, then remove it from <code><a href="#widl-MediaSource-activeSourceBuffers">activeSourceBuffers</a></code> and <a href="http://www.w3.org/TR/html5/webappapis.html#queue-a-task">queue a task</a> to <a href="http://www.w3.org/TR/html5/webappapis.html#fire-a-simple-event">fire a simple event</a> named <code><a href="#dom-evt-removesourcebuffer">removesourcebuffer</a></code> at <code><a href="#widl-MediaSource-activeSourceBuffers">activeSourceBuffers</a></code>.</li>
+ <li>Remove all the tracks in <var>sourceBuffer</var>.<code><a href="#widl-SourceBuffer-audioTracks">audioTracks</a></code>, <var>sourceBuffer</var>.<code><a href="#widl-SourceBuffer-videoTracks">videoTracks</a></code>, and <var>sourceBuffer</var>.<code><a href="#widl-SourceBuffer-textTracks">textTracks</a></code> and <a href="http://www.w3.org/TR/html5/webappapis.html#queue-a-task">queue a task</a> to <a href="http://www.w3.org/TR/html5/webappapis.html#fire-a-simple-event">fire a simple event</a> named <code><a href="http://www.w3.org/TR/html5/embedded-content-0.html#handler-tracklist-onremovetrack">removetrack</a></code> at the modified lists.</li>
+ <li><a href="http://www.w3.org/TR/html5/webappapis.html#queue-a-task">Queue a task</a> to <a href="http://www.w3.org/TR/html5/webappapis.html#fire-a-simple-event">fire a simple event</a> named <code><a href="http://www.w3.org/TR/html5/embedded-content-0.html#handler-tracklist-onremovetrack">removetrack</a></code> at the HTMLMediaElement track lists that were modified.</li>
+ <li>If <var>sourceBuffer</var> is in <code><a href="#widl-MediaSource-activeSourceBuffers">activeSourceBuffers</a></code>, then run the following steps:
+ <ol>
+ <li>Remove <var>sourceBuffer</var> from <code><a href="#widl-MediaSource-activeSourceBuffers">activeSourceBuffers</a></code>.</li>
+ <li><a href="http://www.w3.org/TR/html5/webappapis.html#queue-a-task">Queue a task</a> to <a href="http://www.w3.org/TR/html5/webappapis.html#fire-a-simple-event">fire a simple event</a> named <code><a href="#dom-evt-removesourcebuffer">removesourcebuffer</a></code> at <code><a href="#widl-MediaSource-activeSourceBuffers">activeSourceBuffers</a></code>.</li>
+ <li>If the <a href="http://www.w3.org/TR/html5/embedded-content-0.html#dom-videotrack-selected">selected video track</a> was removed from the <code><a href="http://www.w3.org/TR/html5/embedded-content-0.html#dom-media-videotracks">videoTracks</a></code>
+ attribute on the HTMLMediaElement in a step above, then <a href="http://www.w3.org/TR/html5/webappapis.html#queue-a-task">queue a task</a> to <a href="http://www.w3.org/TR/html5/webappapis.html#fire-a-simple-event">fire a simple event</a> named
+ <code><a href="http://www.w3.org/TR/html5/embedded-content-0.html#handler-tracklist-onchange">change</a></code> at the <code><a href="http://www.w3.org/TR/html5/embedded-content-0.html#dom-media-videotracks">videoTracks</a></code> attribute.</li>
+ <li>If an <a href="http://www.w3.org/TR/html5/embedded-content-0.html#dom-audiotrack-enabled">enabled audio track</a> was removed from the <code><a href="http://www.w3.org/TR/html5/embedded-content-0.html#dom-media-audiotracks">audioTracks</a></code>
+ attribute on the HTMLMediaElement in a step above, then <a href="http://www.w3.org/TR/html5/webappapis.html#queue-a-task">queue a task</a> to <a href="http://www.w3.org/TR/html5/webappapis.html#fire-a-simple-event">fire a simple event</a> named
+ <code><a href="http://www.w3.org/TR/html5/embedded-content-0.html#handler-tracklist-onchange">change</a></code> at the <code><a href="http://www.w3.org/TR/html5/embedded-content-0.html#dom-media-audiotracks">audioTracks</a></code> attribute.</li>
+ <li>If a <code><a href="http://www.w3.org/TR/html5/embedded-content-0.html#texttrack">TextTrack</a></code> with its <code><a href="http://www.w3.org/TR/html5/embedded-content-0.html#dom-texttrack-mode">mode</a></code> attribute set to
+ <a href="http://www.w3.org/TR/html5/embedded-content-0.html#dom-texttrack-showing">"showing"</a> or <a href="http://www.w3.org/TR/html5/embedded-content-0.html#dom-texttrack-hidden">"hidden"</a>
+ was removed from the <code><a href="http://www.w3.org/TR/html5/embedded-content-0.html#dom-media-texttracks">textTracks</a></code> attribute on the HTMLMediaElement in a step above, then
+ <a href="http://www.w3.org/TR/html5/webappapis.html#queue-a-task">queue a task</a> to <a href="http://www.w3.org/TR/html5/webappapis.html#fire-a-simple-event">fire a simple event</a> named <code><a href="http://www.w3.org/TR/html5/embedded-content-0.html#handler-tracklist-onchange">change</a></code> at the <code><a href="http://www.w3.org/TR/html5/embedded-content-0.html#dom-media-texttracks">textTracks</a></code> attribute.</li>
+ </ol>
+ </li>
<li>Remove <var>sourceBuffer</var> from <code><a href="#widl-MediaSource-sourceBuffers">sourceBuffers</a></code> and <a href="http://www.w3.org/TR/html5/webappapis.html#queue-a-task">queue a task</a> to <a href="http://www.w3.org/TR/html5/webappapis.html#fire-a-simple-event">fire a simple event</a> named <code><a href="#dom-evt-removesourcebuffer">removesourcebuffer</a></code> at <code><a href="#widl-MediaSource-sourceBuffers">sourceBuffers</a></code>.</li>
<li>Destroy all resources for <var>sourceBuffer</var>.</li>
</ol></dd></dl></section>
@@ -1085,11 +1100,17 @@
<ol>
<li>If this object has been removed from the <code><a href="#widl-MediaSource-sourceBuffers">sourceBuffers</a></code> attribute of the <a href="#parent-media-source">parent media source</a>, then throw an <code><a href="http://dom.spec.whatwg.org/#dom-domexception-invalid_state_err">INVALID_STATE_ERR</a></code> exception and abort these steps.</li>
<li>If the <code><a href="#widl-SourceBuffer-updating">updating</a></code> attribute equals true, then throw an <code><a href="http://dom.spec.whatwg.org/#dom-domexception-invalid_state_err">INVALID_STATE_ERR</a></code> exception and abort these steps.</li>
- <li>If the <code><a href="#widl-MediaSource-readyState">readyState</a></code> attribute of the <a href="#parent-media-source">parent media source</a> is not in the <code><a href="#idl-def-ReadyState">"open"</a></code> state, then throw an <code><a href="http://dom.spec.whatwg.org/#dom-domexception-invalid_state_err">INVALID_STATE_ERR</a></code> exception and abort these steps.</li>
+ <li>If the <code><a href="#widl-MediaSource-readyState">readyState</a></code> attribute of the <a href="#parent-media-source">parent media source</a> is in the <code><a href="#idl-def-ReadyState">"closed"</a></code> state then throw an <code><a href="http://dom.spec.whatwg.org/#dom-domexception-invalid_state_err">INVALID_STATE_ERR</a></code> exception and abort these steps.</li>
+ <li>
+ <p>If the <code><a href="#widl-MediaSource-readyState">readyState</a></code> attribute of the <a href="#parent-media-source">parent media source</a> is in the <code><a href="#idl-def-ReadyState">"ended"</a></code> state then run the following steps:</p>
+ <ol>
+ <li>Set the <code><a href="#widl-MediaSource-readyState">readyState</a></code> attribute of the <a href="#parent-media-source">parent media source</a> to <code><a href="#idl-def-ReadyState">"open"</a></code></li>
+ <li><a href="http://www.w3.org/TR/html5/webappapis.html#queue-a-task">Queue a task</a> to <a href="http://www.w3.org/TR/html5/webappapis.html#fire-a-simple-event">fire a simple event</a> named <code><a href="#dom-evt-sourceopen">sourceopen</a></code> at the <a href="#parent-media-source">parent media source</a>.</li>
+ </ol>
+ </li>
<li>If this object is waiting for the end of a <a href="#media-segment">media segment</a> to be appended, then throw an <code><a href="http://dom.spec.whatwg.org/#dom-domexception-invalid_state_err">INVALID_STATE_ERR</a></code> and abort these steps.</li>
<li>Update the attribute to the new value.</li>
</ol>
- <div class="issue"><div class="issue-title"><span>Issue 4</span></div><p class=""><a href="https://www.w3.org/Bugs/Public/show_bug.cgi?id=19676">Bug 19676</a> - timestampOffset accuracy</p></div>
</dd><dt id="widl-SourceBuffer-updating"><code>updating</code> of type <span class="idlAttrType"><a>boolean</a></span>, readonly</dt><dd>
<p>Indicates whether an <code><a href="#widl-SourceBuffer-appendArrayBuffer-void-ArrayBuffer-data">appendArrayBuffer()</a></code>, <code><a href="#widl-SourceBuffer-appendStream-void-Stream-stream-unsigned-long-long-maxSize">appendStream()</a></code>, or <code><a href="#widl-SourceBuffer-remove-void-double-start-double-end">remove()</a></code> operation is still being
processed.</p>
@@ -1137,6 +1158,10 @@
<li>Set the <code><a href="#widl-SourceBuffer-updating">updating</a></code> attribute to true.</li>
<li><a href="http://www.w3.org/TR/html5/webappapis.html#queue-a-task">Queue a task</a> to <a href="http://www.w3.org/TR/html5/webappapis.html#fire-a-simple-event">fire a simple event</a> named <code><a href="#dom-evt-updatestart">updatestart</a></code> at this <a href="#idl-def-SourceBuffer" class="idlType"><code>SourceBuffer</code></a> object.</li>
<li>Asynchronously run the <a href="#sourcebuffer-segment-parser-loop">segment parser loop</a> algorithm.</li>
+ <li>When the <a href="#sourcebuffer-segment-parser-loop">segment parser loop</a> returns control to this algorithm, run the remaining steps.</li>
+ <li>Set the <code><a href="#widl-SourceBuffer-updating">updating</a></code> attribute to false.</li>
+ <li><a href="http://www.w3.org/TR/html5/webappapis.html#queue-a-task">Queue a task</a> to <a href="http://www.w3.org/TR/html5/webappapis.html#fire-a-simple-event">fire a simple event</a> named <code><a href="#dom-evt-update">update</a></code> at this <a href="#idl-def-SourceBuffer" class="idlType"><code>SourceBuffer</code></a> object.</li>
+ <li><a href="http://www.w3.org/TR/html5/webappapis.html#queue-a-task">Queue a task</a> to <a href="http://www.w3.org/TR/html5/webappapis.html#fire-a-simple-event">fire a simple event</a> named <code><a href="#dom-evt-updateend">updateend</a></code> at this <a href="#idl-def-SourceBuffer" class="idlType"><code>SourceBuffer</code></a> object.</li>
</ol></dd><dt id="widl-SourceBuffer-appendStream-void-Stream-stream-unsigned-long-long-maxSize"><code>appendStream</code></dt><dd>
<p>Appends segment data to the source buffer from a <a href="http://dvcs.w3.org/hg/streams-api/raw-file/tip/Overview.htm#idl-def-Stream">Stream</a>.</p>
@@ -1196,6 +1221,7 @@
<li>If <var><a href="#sourcebuffer-buffer-full-flag">buffer full flag</a></var> equals true and this object is ready to accept more bytes, then set
the <var><a href="#sourcebuffer-buffer-full-flag">buffer full flag</a></var> to false.</li>
<li>Set the <code><a href="#widl-SourceBuffer-updating">updating</a></code> attribute to false.</li>
+ <li><a href="http://www.w3.org/TR/html5/webappapis.html#queue-a-task">Queue a task</a> to <a href="http://www.w3.org/TR/html5/webappapis.html#fire-a-simple-event">fire a simple event</a> named <code><a href="#dom-evt-update">update</a></code> at this <a href="#idl-def-SourceBuffer" class="idlType"><code>SourceBuffer</code></a> object.</li>
<li><a href="http://www.w3.org/TR/html5/webappapis.html#queue-a-task">Queue a task</a> to <a href="http://www.w3.org/TR/html5/webappapis.html#fire-a-simple-event">fire a simple event</a> named <code><a href="#dom-evt-updateend">updateend</a></code> at this <a href="#idl-def-SourceBuffer" class="idlType"><code>SourceBuffer</code></a> object.</li>
</ol></dd></dl></section>
@@ -1231,19 +1257,24 @@
<td><code><a href="#widl-SourceBuffer-updating">updating</a></code> transitions from false to true.</td>
</tr>
<tr>
+ <td><dfn id="dom-evt-update"><code>update</code></dfn></td>
+ <td><code>Event</code></td>
+ <td>The append or remove has successfully completed. <code><a href="#widl-SourceBuffer-updating">updating</a></code> transitions from true to false.</td>
+ </tr>
+ <tr>
<td><dfn id="dom-evt-updateend"><code>updateend</code></dfn></td>
<td><code>Event</code></td>
- <td><code><a href="#widl-SourceBuffer-updating">updating</a></code> transitions from true to false.</td>
+ <td>The append or remove has ended.</td>
</tr>
<tr>
<td><dfn id="dom-evt-error"><code>error</code></dfn></td>
<td><code>Event</code></td>
- <td>An error occurred during the append.</td>
+ <td>An error occurred during the append. <code><a href="#widl-SourceBuffer-updating">updating</a></code> transitions from true to false.</td>
</tr>
<tr>
<td><dfn id="dom-evt-abort"><code>abort</code></dfn></td>
<td><code>Event</code></td>
- <td>The append or remove was aborted by an <code><a href="#widl-SourceBuffer-abort-void">abort()</a></code> call.</td>
+ <td>The append or remove was aborted by an <code><a href="#widl-SourceBuffer-abort-void">abort()</a></code> call. <code><a href="#widl-SourceBuffer-updating">updating</a></code> transitions from true to false.</td>
</tr>
</tbody>
</table>
@@ -1326,9 +1357,7 @@
<li>Jump to the <i>loop top</i> step above.</li>
</ol>
</li>
- <li><i>Need more data:</i> If the <a href="#sourcebuffer-stream-append-loop">stream append loop</a> algorithm is running and still has data to read, then abort these steps.
- </li><li>Set the <code><a href="#widl-SourceBuffer-updating">updating</a></code> attribute to false.</li>
- <li><a href="http://www.w3.org/TR/html5/webappapis.html#queue-a-task">Queue a task</a> to <a href="http://www.w3.org/TR/html5/webappapis.html#fire-a-simple-event">fire a simple event</a> named <code><a href="#dom-evt-updateend">updateend</a></code> at this <a href="#idl-def-SourceBuffer" class="idlType"><code>SourceBuffer</code></a> object.</li>
+ <li><i>Need more data:</i> Return control to the calling algorithm.</li>
</ol>
</section>
@@ -1353,7 +1382,7 @@
<li>Set the <code><a href="#widl-SourceBuffer-updating">updating</a></code> attribute to false.</li>
<li>
<a href="http://www.w3.org/TR/html5/webappapis.html#queue-a-task">Queue a task</a> to <a href="http://www.w3.org/TR/html5/webappapis.html#fire-a-simple-event">fire a simple event</a> named <code><a href="#dom-evt-error">error</a></code> at this <a href="#idl-def-SourceBuffer" class="idlType"><code>SourceBuffer</code></a> object.
- <div class="issue"><div class="issue-title"><span>Issue 5</span></div><p class="">Need a way to convey error information.</p></div>
+ <div class="issue"><div class="issue-title"><span>Issue 4</span></div><p class="">Need a way to convey error information.</p></div>
</li>
<li><a href="http://www.w3.org/TR/html5/webappapis.html#queue-a-task">Queue a task</a> to <a href="http://www.w3.org/TR/html5/webappapis.html#fire-a-simple-event">fire a simple event</a> named <code><a href="#dom-evt-updateend">updateend</a></code> at this <a href="#idl-def-SourceBuffer" class="idlType"><code>SourceBuffer</code></a> object.</li>
</ol>
@@ -1391,6 +1420,7 @@
<li>Run the <a href="#sourcebuffer-segment-parser-loop">segment parser loop</a> algorithm.</li>
<li>Jump to the <i>loop top</i> step above.</li>
<li><i>Loop Done: </i>Set the <code><a href="#widl-SourceBuffer-updating">updating</a></code> attribute to false.</li>
+ <li><a href="http://www.w3.org/TR/html5/webappapis.html#queue-a-task">Queue a task</a> to <a href="http://www.w3.org/TR/html5/webappapis.html#fire-a-simple-event">fire a simple event</a> named <code><a href="#dom-evt-update">update</a></code> at this <a href="#idl-def-SourceBuffer" class="idlType"><code>SourceBuffer</code></a> object.</li>
<li><a href="http://www.w3.org/TR/html5/webappapis.html#queue-a-task">Queue a task</a> to <a href="http://www.w3.org/TR/html5/webappapis.html#fire-a-simple-event">fire a simple event</a> named <code><a href="#dom-evt-updateend">updateend</a></code> at this <a href="#idl-def-SourceBuffer" class="idlType"><code>SourceBuffer</code></a> object.</li>
</ol>
</section>
@@ -1443,7 +1473,12 @@
</ol>
</li>
<li>Add <var>new audio track</var> to the <code><a href="#widl-SourceBuffer-audioTracks">audioTracks</a></code> attribute on this <a href="#idl-def-SourceBuffer" class="idlType"><code>SourceBuffer</code></a> object.</li>
+ <li><a href="http://www.w3.org/TR/html5/webappapis.html#queue-a-task">Queue a task</a> to <a href="http://www.w3.org/TR/html5/webappapis.html#fire-a-simple-event">fire a simple event</a> named <code><a href="http://www.w3.org/TR/html5/embedded-content-0.html#handler-tracklist-onaddtrack">addtrack</a></code> at
+ <code><a href="#widl-SourceBuffer-audioTracks">audioTracks</a></code> attribute
+ on this <a href="#idl-def-SourceBuffer" class="idlType"><code>SourceBuffer</code></a> object.</li>
<li>Add <var>new audio track</var> to the <code><a href="http://www.w3.org/TR/html5/embedded-content-0.html#dom-media-audiotracks">audioTracks</a></code> attribute on the HTMLMediaElement.</li>
+ <li><a href="http://www.w3.org/TR/html5/webappapis.html#queue-a-task">Queue a task</a> to <a href="http://www.w3.org/TR/html5/webappapis.html#fire-a-simple-event">fire a simple event</a> named <code><a href="http://www.w3.org/TR/html5/embedded-content-0.html#handler-tracklist-onaddtrack">addtrack</a></code> at the <code><a href="http://www.w3.org/TR/html5/embedded-content-0.html#dom-media-audiotracks">audioTracks</a></code>
+ attribute on the HTMLMediaElement.</li>
<li>Create a new <a href="#track-buffer">track buffer</a> to store <a href="#coded-frame">coded frames</a> for this track.</li>
<li>Add the <a href="#track-description">track description</a> for this track to the <a href="#track-buffer">track buffer</a>.</li>
</ol>
@@ -1464,7 +1499,11 @@
</ol>
</li>
<li>Add <var>new video track</var> to the <code><a href="#widl-SourceBuffer-videoTracks">videoTracks</a></code> attribute on this <a href="#idl-def-SourceBuffer" class="idlType"><code>SourceBuffer</code></a> object.</li>
+ <li><a href="http://www.w3.org/TR/html5/webappapis.html#queue-a-task">Queue a task</a> to <a href="http://www.w3.org/TR/html5/webappapis.html#fire-a-simple-event">fire a simple event</a> named <code><a href="http://www.w3.org/TR/html5/embedded-content-0.html#handler-tracklist-onaddtrack">addtrack</a></code> at <code><a href="#widl-SourceBuffer-videoTracks">videoTracks</a></code> attribute
+ on this <a href="#idl-def-SourceBuffer" class="idlType"><code>SourceBuffer</code></a> object.</li>
<li>Add <var>new video track</var> to the <code><a href="http://www.w3.org/TR/html5/embedded-content-0.html#dom-media-videotracks">videoTracks</a></code> attribute on the HTMLMediaElement.</li>
+ <li><a href="http://www.w3.org/TR/html5/webappapis.html#queue-a-task">Queue a task</a> to <a href="http://www.w3.org/TR/html5/webappapis.html#fire-a-simple-event">fire a simple event</a> named <code><a href="http://www.w3.org/TR/html5/embedded-content-0.html#handler-tracklist-onaddtrack">addtrack</a></code> at the <code><a href="http://www.w3.org/TR/html5/embedded-content-0.html#dom-media-videotracks">videoTracks</a></code> attribute on the
+ HTMLMediaElement.</li>
<li>Create a new <a href="#track-buffer">track buffer</a> to store <a href="#coded-frame">coded frames</a> for this track.</li>
<li>Add the <a href="#track-description">track description</a> for this track to the <a href="#track-buffer">track buffer</a>.</li>
</ol>
@@ -1480,7 +1519,11 @@
<code><a href="http://www.w3.org/TR/html5/embedded-content-0.html#dom-texttrack-hidden">"hidden"</a></code>, then set <var>active track flag</var> to true.
</li>
<li>Add <var>new text track</var> to the <code><a href="#widl-SourceBuffer-textTracks">textTracks</a></code> attribute on this <a href="#idl-def-SourceBuffer" class="idlType"><code>SourceBuffer</code></a> object.</li>
+ <li><a href="http://www.w3.org/TR/html5/webappapis.html#queue-a-task">Queue a task</a> to <a href="http://www.w3.org/TR/html5/webappapis.html#fire-a-simple-event">fire a simple event</a> named <code><a href="http://www.w3.org/TR/html5/embedded-content-0.html#handler-tracklist-onaddtrack">addtrack</a></code> at <code><a href="#widl-SourceBuffer-textTracks">textTracks</a></code> attribute
+ on this <a href="#idl-def-SourceBuffer" class="idlType"><code>SourceBuffer</code></a> object.</li>
<li>Add <var>new text track</var> to the <code><a href="http://www.w3.org/TR/html5/embedded-content-0.html#dom-media-texttracks">textTracks</a></code> attribute on the HTMLMediaElement.</li>
+ <li><a href="http://www.w3.org/TR/html5/webappapis.html#queue-a-task">Queue a task</a> to <a href="http://www.w3.org/TR/html5/webappapis.html#fire-a-simple-event">fire a simple event</a> named <code><a href="http://www.w3.org/TR/html5/embedded-content-0.html#handler-tracklist-onaddtrack">addtrack</a></code> at the <code><a href="http://www.w3.org/TR/html5/embedded-content-0.html#dom-media-texttracks">textTracks</a></code> attribute on the
+ HTMLMediaElement.</li>
<li>Create a new <a href="#track-buffer">track buffer</a> to store <a href="#coded-frame">coded frames</a> for this track.</li>
<li>Add the <a href="#track-description">track description</a> for this track to the <a href="#track-buffer">track buffer</a>.</li>
</ol>
@@ -1518,8 +1561,8 @@
<li>
<p>For each <a href="#coded-frame">coded frame</a> in the <a href="#media-segment">media segment</a> run the following steps:</p>
<ol>
- <li>Let <var>presentation timestamp</var> be a double precision floating point representation of the coded frame's presentation timestamp.</li>
- <li>Let <var>decode timestamp</var> be a double precision floating point representation of the coded frame's decode timestamp.
+ <li>Let <var>presentation timestamp</var> be a double precision floating point representation of the coded frame's presentation timestamp in seconds.</li>
+ <li>Let <var>decode timestamp</var> be a double precision floating point representation of the coded frame's decode timestamp in seconds.
<div class="note"><div class="note-title"><span>Note</span></div><p class="">Implementations don't have to internally store timestamps in a double precision floating point representation. This
representation is used here because it is the represention for timestamps in the HTML spec. The intention here is to make the
behavior clear without adding unnecessary complexity to the algorithm to deal with the fact that adding a timestampOffset may
@@ -1528,6 +1571,7 @@
if a double precision floating point representation was used.
</p></div>
</li>
+ <li>Let <var>frame duration</var> be a double precision floating point representation of the coded frame's duration in seconds.</li>
<li>
<p>If <code><a href="#widl-SourceBuffer-timestampOffset">timestampOffset</a></code> is not 0, then run the following steps:</p>
<ol>
@@ -1544,25 +1588,52 @@
<code><a href="#widl-SourceBuffer-abort-void">abort()</a></code>.</p></div>
</li>
- <li>If <var><a href="#highest-presentation-timestamp">highest presentation timestamp</a></var> for <var>track buffer</var> is set and less than <var>presentation timestamp</var>,
- then run the following steps:
+ <li>Let <var>frame end timestamp</var> equal the sum of <var>presentation timestamp</var> and <var>frame duration</var>.</li>
+ <li>Let <var>spliced frame</var> be an unset variable for holding audio splice information</li>
+ <li>If <var><a href="#last-decode-timestamp">last decode timestamp</a></var> for <var>track buffer</var> is unset and <var>presentation timestamp</var> lies within a
+ <a href="#coded-frame">coded frame</a> already stored in <var>track buffer</var>, then run the following steps:
<ol>
- <li>Remove all <a href="#coded-frame">coded frames</a> from <var>track buffer</var> that have a presentation timestamp greater than
- <var><a href="#highest-presentation-timestamp">highest presentation timestamp</a></var> and less than or equal to <var>presentation timestamp</var>.</li>
- <li>Remove all <a href="#coded-frame">coded frames</a> from <var>track buffer</var> that have decoding dependencies on the coded frames removed in
- the previous step.
- <div class="note"><div class="note-title"><span>Note</span></div><p class="">For example if an I-frame is removed in the previous step, then all P-frames & B-frames that depend on that I-frame
- should be removed from <var>track buffer</var>. This makes sure that decode dependencies are properly maintained during overlaps.</p></div>
+ <li>Let <var>overlapped frame</var> be the <a href="#coded-frame">coded frame</a> in <var>track buffer</var> that contains <var>presentation timestamp</var>.</li>
+ <li>If <var>track buffer</var> contains audio <a href="#coded-frame">coded frames</a>, then run the <a href="#sourcebuffer-audio-splice-frame-algorithm">audio splice frame algorithm</a> and if a splice
+ frame is returned, assign it to <var>spliced frame</var>.</li>
+ <li>If <var>track buffer</var> contains video <a href="#coded-frame">coded frames</a> and <var>presentation timestamp</var> is less than 1 microsecond
+ beyond the presentation timestamp of <var>overlapped frame</var>, then remove <var>overlapped frame</var> and any
+ <a href="#coded-frame">coded frames</a> that depend on it from <var>track buffer</var>.
+ <div class="note"><div class="note-title"><span>Note</span></div><p class="">
+ This is to compensate for minor errors in frame timestamp computations that can appear when converting back and forth between double precision
+ floating point numbers and rationals. This tolerance allows a frame to replace an existing one as long as it is within 1 microsecond of the existing
+ frame's start time. Frames that come slightly before an existing frame are handled by the removal step below.
+ </p></div>
</li>
-
</ol>
</li>
- <li>Add the <a href="#coded-frame">coded frame</a> with the <var>presentation timestamp</var> and <var>decode timestamp</var>, to the
- <var>track buffer</var>.</li>
- <li>Set <var><a href="#last-decode-timestamp">last decode timestamp</a></var> for <var>track buffer</var> to <var>decode timestamp</var>.</li>
- <li>If <var><a href="#highest-presentation-timestamp">highest presentation timestamp</a></var> for <var>track buffer</var> is unset or <var>presentation timestamp</var> is greater
+ <li>Remove existing coded frames in <var>track buffer</var>:
+ <dl class="switch">
+ <dt>If <var><a href="#highest-presentation-timestamp">highest presentation timestamp</a></var> for <var>track buffer</var> is not set:</dt>
+ <dd>Remove all <a href="#coded-frame">coded frames</a> from <var>track buffer</var> that have a presentation timestamp greater than or equal to
+ <var>presentation timestamp</var> and less than <var>frame end timestamp</var>.</dd>
+ <dt>If <var><a href="#highest-presentation-timestamp">highest presentation timestamp</a></var> for <var>track buffer</var> is set and less than <var>presentation timestamp</var></dt>
+ <dd>Remove all <a href="#coded-frame">coded frames</a> from <var>track buffer</var> that have a presentation timestamp greater than
+ <var><a href="#highest-presentation-timestamp">highest presentation timestamp</a></var> and less than or equal to <var>frame end timestamp</var>.</dd>
+ </dl>
+ </li>
+ <li>Remove all <a href="#coded-frame">coded frames</a> from <var>track buffer</var> that have decoding dependencies on the coded frames removed in
+ the previous step.
+ <div class="note"><div class="note-title"><span>Note</span></div><p class="">For example if an I-frame is removed in the previous step, then all P-frames & B-frames that depend on that I-frame
+ should be removed from <var>track buffer</var>. This makes sure that decode dependencies are properly maintained during overlaps.</p></div>
+ </li>
+ <li>
+ <dl class="switch">
+ <dt>If <var>spliced frame</var> is set:</dt>
+ <dd>Add <var>spliced frame</var> to the <var>track buffer</var>.</dd>
+ <dt>Otherwise:</dt>
+ <dd>Add the <a href="#coded-frame">coded frame</a> with the <var>presentation timestamp</var>, <var>decode timestamp</var>, and <var>frame duration</var> to the
+ <var>track buffer</var>.</dd>
+ </dl>
+ </li><li>Set <var><a href="#last-decode-timestamp">last decode timestamp</a></var> for <var>track buffer</var> to <var>decode timestamp</var>.</li>
+ <li>If <var><a href="#highest-presentation-timestamp">highest presentation timestamp</a></var> for <var>track buffer</var> is unset or <var>frame end timestamp</var> is greater
than <var><a href="#highest-presentation-timestamp">highest presentation timestamp</a></var>, then set <var><a href="#highest-presentation-timestamp">highest presentation timestamp</a></var> for <var>track buffer</var>
- to <var>presentation timestamp</var>.
+ to <var>frame end timestamp</var>.
<div class="note"><div class="note-title"><span>Note</span></div><p class="">The greater than check is needed because bidirectional prediction between coded frames can cause
<var>presentation timestamp</var> to not be monotonically increasing eventhough the decode timestamps are monotonically increasing.</p></div>
</li>
@@ -1594,6 +1665,97 @@
<li>If the <a href="#media-segment">media segment</a> contains data beyond the current <code><a href="#widl-MediaSource-duration">duration</a></code>, then run the <a href="#duration-change-algorithm">duration change algorithm</a> with <var>new duration</var> set to the maximum of the current duration and the highest end timestamp reported by <code><a href="http://www.w3.org/TR/html5/embedded-content-0.html#dom-media-buffered">HTMLMediaElement.buffered</a></code>.</li>
</ol>
</section>
+
+ <section id="sourcebuffer-audio-splice-frame-algorithm">
+ <h4><span class="secno">4.5.7 </span>Audio Splice Frame Algorithm</h4>
+ <p>Follow these steps when the <a href="#sourcebuffer-coded-frame-processing">coded frame processing algorithm</a> needs to generate a splice frame for two overlapping audio
+ <a href="#coded-frame">coded frames</a>:</p>
+ <ol>
+ <li>Let <var>track buffer</var> be the <a href="#track-buffer">track buffer</a> that will contain the splice.</li>
+ <li>Let <var>new coded frame</var> be the new <a href="#coded-frame">coded frame</a>, that is being added to <var>track buffer</var>, which triggered the need for a splice.</li>
+ <li>Let <var>presentation timestamp</var> be the presentation timestamp for <var>new coded frame</var></li>
+ <li>Let <var>decode timestamp</var> be the decode timestamp for <var>new coded frame</var>.</li>
+ <li>Let <var>frame duration</var> be the duration of <var>new coded frame</var>.</li>
+ <li>Let <var>overlapped frame</var> be the <a href="#coded-frame">coded frame</a> in <var>track buffer</var> that overlaps with <var>new coded frame</var>
+ (ie. it contains <var>presentation timestamp).</var></li>
+ <li>Round & update <var>presentation timestamp</var> and <var>decode timestamp</var> to the nearest audio sample timestamp based on sample rate of the audio in <var>overlapped frame</var>.
+ <div class="note"><div class="note-title"><span>Note</span></div><div class="">
+ <p>For example, given the following values:</p>
+ <ul>
+ <li>The presentation timestamp of <var>overlapped frame</var> equals 10.</li>
+ <li>The sample rate of <var>overlapped frame</var> equals 8000 Hz</li>
+ <li><var>presentation timestamp</var> equals 10.01255</li>
+ <li><var>decode timestamp</var> equals 10.01255</li>
+ </ul>
+ <p><var>presentation timestamp</var> and <var>decode timestamp</var> are rounded & updated to 10.0125 since 10.01255 is closer to
+ 10 + 100/8000 (10.0125) than 10 + 101/8000 (10.012625)</p>
+ </div></div>
+ </li>
+ <li>If the user agent does not support crossfading then run the following steps:
+ <ol>
+ <li>Remove <var>overlapped frame</var> from <var>track buffer</var>.</li>
+ <li>Add a silence frame to <var>track buffer</var> with the following properties:
+ <ul>
+ <li>The presentation time set to the <var>overlapped frame</var> presentation time.</li>
+ <li>The decode time set to the <var>overlapped frame</var> decode time.</li>
+ <li>The frame duration set to difference between <var>presentation timestamp</var> and the <var>overlapped frame</var> presentation time.</li>
+ </ul>
+ </li>
+ <li>Return to caller without providing a splice frame.
+ <div class="note"><div class="note-title"><span>Note</span></div><p class="">
+ This is intended to allow <var>new coded frame</var> to be added to the <var>track buffer</var> as if
+ <var>overlapped frame</var> had not been in the <var>track buffer</var> to begin with.
+ </p></div>
+ </li>
+ </ol>
+ </li>
+ <li>Let <var>frame end timestamp</var> equal the sum of <var>presentation timestamp</var> and <var>frame duration</var>.</li>
+ <li>Let <var>fade out coded frames</var> equal <var>overlapped frame</var> as well as any addition frames in <var>track buffer</var> that overlap
+ <var>presentation timestamp</var> plus the splice duration of 5 milliseconds.</li>
+ <li>Remove all the frames included in <var>fade out coded frames</var> from <var>track buffer</var>.
+ </li><li>Return a splice frame with the following properties:
+ <ul>
+ <li>The presentation time set to the <var>overlapped frame</var> presentation time.</li>
+ <li>The decode time set to the <var>overlapped frame</var> decode time.</li>
+ <li>The frame duration set to difference between <var>frame end timestamp</var> and the <var>overlapped frame</var> presentation time.</li>
+ <li>The fade out coded frames equals <var>fade-out coded frames</var>.</li>
+ <li>The fade in coded frame equals <var>new coded frame</var>.</li>
+ <li>The splice timestamp equals <var>presentation timestamp</var>.</li>
+ </ul>
+ <div class="note"><div class="note-title"><span>Note</span></div><p class="">See the <a href="#sourcebuffer-audio-splice-rendering-algorithm">audio splice rendering algorithm</a> for details on how this splice frame is rendered.</p></div>
+ </li>
+ </ol>
+ </section>
+ <section id="sourcebuffer-audio-splice-rendering-algorithm">
+ <h4><span class="secno">4.5.8 </span>Audio Splice Rendering Algorithm</h4>
+ <p>The following steps are run when a spliced frame, generated by the <a href="#sourcebuffer-audio-splice-frame-algorithm">audio splice frame algorithm</a>, needs to be rendered by the
+ media element:</p>
+ <ol>
+ <li>Let <var>fade out coded frames</var> be the <a href="#coded-frame">coded frames</a> that are faded out during the splice.</li>
+ <li>Let <var>fade in coded frames</var> be the <a href="#coded-frame">coded frames</a> that are faded in during the splice.</li>
+ <li>Let <var>presentation timestamp</var> be the presentation timestamp of the first coded frame in <var>fade out coded frames</var>.</li>
+ <li>Let <var>end timestamp</var> be the sum of the presentation timestamp and frame duration in the last frame in <var>fade in coded frames</var>.</li>
+ <li>Let <var>splice timestamp</var> be the presentation timestamp where the splice starts. This corresponds with the presentation timestamp of the first frame in
+ <var>fade in coded frames</var>.</li>
+ <li>Let <var>splice end timestamp</var> equal <var>splice timestamp</var> plus five milliseconds.</li>
+ <li>Let <var>fade out samples</var> be the samples generated by decoding <var>fade out coded frames</var>.</li>
+ <li>Trim <var>fade out samples</var> so that it only contains samples between <var>presentation timestamp</var> and <var>splice end timestamp</var>.</li>
+ <li>Let <var>fade in samples</var> be the samples generated by decoding <var>fade in coded frames</var>.</li>
+ <li>Convert <var>fade out samples</var> and <var>fade in samples</var> to a common sample rate and channel layout.</li>
+ <li>Let <var>output samples</var> be a buffer to hold the output samples.</li>
+ <li>Apply a linear gain fade out to the samples between <var>splice timestamp</var> and <var>splice end timestamp</var> in <var>fade out samples</var>.</li>
+ <li>Apply a linear gain fade in to the samples between <var>splice timestamp</var> and <var>splice end timestamp</var> in <var>fade in samples</var>.</li>
+ <li>Copy samples between <var>presentation timestamp</var> to <var>splice timestamp</var> from <var>fade out samples</var> into <var>output samples</var>.</li>
+ <li>For each sample between <var>splice timestamp</var> and <var>splice end timestamp</var>, compute the sum of a sample from <var>fade out samples</var> and the
+ corresponding sample in <var>fade in samples</var> and store the result in <var>output samples</var>.</li>
+ <li>Copy samples between <var>splice end timestamp</var> to <var>end timestamp</var> from <var>fade in samples</var> into <var>output samples</var>.</li>
+ <li>Render <var>output samples</var>.</li>
+ </ol>
+ <div class="note"><div class="note-title"><span>Note</span></div><div class="">
+ <p>Here is a graphical representation of this algorithm.</p>
+ <img src="audio_splice.png" alt="Audio splice diagram">
+ </div></div>
+ </section>
</section>
</section>
@@ -1691,8 +1853,6 @@
</li>
<li>Return the <var>intersection ranges</var>.</li>
</ol>
- <div class="issue"><div class="issue-title"><span>Issue 6</span></div><p class=""><a href="https://www.w3.org/Bugs/Public/show_bug.cgi?id=18615">Bug 18615</a> - Define how SourceBuffer.buffered maps to HTMLMediaElement.buffered</p></div>
- <div class="issue"><div class="issue-title"><span>Issue 7</span></div><p class=""><a href="https://www.w3.org/Bugs/Public/show_bug.cgi?id=18400">Bug 18400</a> - Define and document timestamp heuristics</p></div>
</section>
<section id="audio-track-extensions">
@@ -2001,7 +2161,6 @@
<li>PSI that is identical to the information in the initialization segment may appear repeatedly throughout the segment.</li>
<li>The media segment will not rely on initialization information in another media segment.</li>
<li>Media Segments must contain only complete PES packets and sections.</li>
- <li>Each PES packet must be comprised of one or more complete access units.</li>
<li>Each PES packet must have a PTS timestamp.</li>
<li>PCR must be present in the Segment prior to the first byte of a TS packet payload containing media data.</li>
<li>The presentation duration of each media component within the Media Segment should be as close to equal as practical.</li>
@@ -2139,7 +2298,20 @@
</thead>
<tbody>
<tr>
- <td>05 February 2013</td>
+ <td>19 February 2013</td>
+ <td>
+ <ul>
+ <li>Bug 19676, 20327 - Provide more detail for audio & video splicing.</li>
+ <li>Bug 20900 - Remove complete access unit constraint.</li>
+ <li>Bug 20948 - Setting timestampOffset in 'ended' triggers a transition to 'open'</li>
+ <li>Bug 20952 - Added update event.</li>
+ <li>Bug 20953 - Move end of append event firing out of segment parser loop.</li>
+ <li>Bug 21034 - Add steps to fire addtrack and removetrack events.</li>
+ </ul>
+ </td>
+ </tr>
+ <tr>
+ <td><a href="https://dvcs.w3.org/hg/html-media/raw-file/77975abeec41/media-source/media-source.html">05 February 2013</a></td>
<td>
<ul>
<li>Bug 19676 - Added a note clarifying that the internal timestamp representation doesn't have to be a double.</li>
@@ -2239,11 +2411,11 @@
<td>Converted document to ReSpec.</td>
</tr>
<tr>
- <td><a href="http://dvcs.w3.org/hg/html-media/rev/e029f71aafca">18 October 2012</a></td>
+ <td><a href="http://dvcs.w3.org/hg/html-media/raw-file/e029f71aafca/media-source/media-source.html">18 October 2012</a></td>
<td>Refactored SourceBuffer.append() & added SourceBuffer.remove().</td>
</tr>
<tr>
- <td><a href="http://dvcs.w3.org/hg/html-media/rev/6d127e69c9f8">8 October 2012</a></td>
+ <td><a href="http://dvcs.w3.org/hg/html-media/raw-file/6d127e69c9f8/media-source/media-source.html">8 October 2012</a></td>
<td>
<ul>
<li>Defined what HTMLMediaElement.seekable and HTMLMediaElement.buffered should return.</li>
--- a/media-source/media-source.js Wed Feb 13 17:04:34 2013 -0500
+++ b/media-source/media-source.js Mon Feb 18 13:44:34 2013 -0800
@@ -93,7 +93,8 @@
'decode': { func: idlref_helper, fragment: 'idl-def-EndOfStreamError', link_text: '"decode"', },
'updatestart': { func: eventref_helper, fragment: 'updatestart', link_text: 'updatestart', },
- 'updateend': { func: eventref_helper, fragment: 'updateend', link_text: 'updateend', },
+ 'update': { func: eventref_helper, fragment: 'update', link_text: 'update', },
+ 'updateend': { func: eventref_helper, fragment: 'updateend', link_text: 'updateend', },
'updateerror': { func: eventref_helper, fragment: 'error', link_text: 'error', },
'updateabort': { func: eventref_helper, fragment: 'abort', link_text: 'abort', },
@@ -133,6 +134,8 @@
'stream-append-loop': { func: link_helper, fragment: '#sourcebuffer-stream-append-loop', link_text: 'stream append loop', },
'init-segment-received-algorithm': { func: link_helper, fragment: '#sourcebuffer-init-segment-received', link_text: 'initialization segment received algorithm', },
'coded-frame-processing-algorithm': { func: link_helper, fragment: '#sourcebuffer-coded-frame-processing', link_text: 'coded frame processing algorithm', },
+ 'audio-splice-frame-algorithm': { func: link_helper, fragment: '#sourcebuffer-audio-splice-frame-algorithm', link_text: 'audio splice frame algorithm', },
+ 'audio-splice-rendering-algorithm': { func: link_helper, fragment: '#sourcebuffer-audio-splice-rendering-algorithm', link_text: 'audio splice rendering algorithm', },
'input-buffer': { func: var_helper, fragment: '#sourcebuffer-input-buffer', link_text: 'input buffer', },
'buffer-full-flag': { func: var_helper, fragment: '#sourcebuffer-buffer-full-flag', link_text: 'buffer full flag', },
'MediaSource-object-URL': { func: link_helper, fragment: '#mediasource-object-url', link_text: 'MediaSource object URL', },
@@ -164,6 +167,8 @@
'timerange': { func: code_videoref_helper, fragment: 'timeranges', link_text: 'TimeRange', },
'timeranges': { func: code_videoref_helper, fragment: 'timeranges', link_text: 'TimeRanges', },
'tracklist-change': { func: code_videoref_helper, fragment: 'handler-tracklist-onchange', link_text: 'change', },
+ 'tracklist-addtrack': { func: code_videoref_helper, fragment: 'handler-tracklist-onaddtrack', link_text: 'addtrack', },
+ 'tracklist-removetrack': { func: code_videoref_helper, fragment: 'handler-tracklist-onremovetrack', link_text: 'removetrack', },
'video-track': { func: code_videoref_helper, fragment: 'videotrack', link_text: 'VideoTrack', },
'videotrack-id': { func: code_videoref_helper, fragment: 'dom-videotrack-id', link_text: 'id', },
'videotrack-kind': { func: code_videoref_helper, fragment: 'dom-videotrack-kind', link_text: 'kind', },