--- a/reqs/Overview.html Fri May 25 08:47:10 2012 +0100
+++ b/reqs/Overview.html Thu May 31 12:08:38 2012 +0100
@@ -150,7 +150,7 @@
<section>
- <h3>UC 3: online music production tool</h3>
+ <h3>UC 3: Online music production tool</h3>
<p>(online audio workstation tool?)
</p>
@@ -247,49 +247,9 @@
<pre> <i>Priority: <b>LOW</b></i>
</pre>
<p>… consensus reached during the teleconference on <a href="http://www.w3.org/2012/02/13-audio-minutes" title="http://www.w3.org/2012/02/13-audio-minutes">13 Feb 2012</a>.
- </p><p>General consensus that while this is an interesting use case, there is no clamor to facilitate it entirely and urgently.
</p>
- <h4>UC5 — Related Requirements </h4>
- <ul><li> <i>Sources of audio</i>
- <ul><li> <b>Support for primary audio file formats</b> (loading of assets in various formats + export)
- </li><li> <b>One source, many sounds</b>
- </li><li> <b>Playing / Looping sources of audio</b> (playback)
- </li><li> <del>Capture of audio from microphone, line in, other inputs</del>
- </li><li> <del>Adding effects to the audio part of a video stream, and keep it in sync with the video playback</del>
- </li><li> <b>Sample-accurate scheduling of playback</b>
- </li><li> <del>Buffering</del>
- </li><li> <b>Support for basic polyphony</b> (many instruments)
- </li><li> <b>Rapid scheduling of many independent sources</b>
- </li><li> <b>Triggering of audio sources</b> (one-shot playback)
- </li><li> <b>Audio quality</b> (playback needs to be good enough to test the score)
- </li></ul>
- </li><li> <i>Transformations of sources of audio</i>
- <ul><li> <b>Modularity of transformations</b> (instrument pitch)
- </li><li> <b>Transformation parameter automation</b>
- </li><li> <b>gain adjustment</b>
- </li><li> <b>playback rate adjustment</b> (choice of tempo in playback)
- </li><li> <del>spatialization</del>
- </li><li> <del>filtering</del>
- </li><li> <del>Noise gating</del>
- </li><li> <del>dynamic range compression</del>
- </li><li> <del>The simulation of acoustic spaces</del>
- </li><li> <del>The simulation of occlusions and obstructions</del>
- </li></ul>
- </li><li> <i>Source Combination and Interaction</i>
- <ul><li> <b>Mixing Sources</b>
- </li><li> <del>Ducking</del>
- </li><li> <del>Echo cancellation</del>
- </li></ul>
- </li><li> <i>Analysis of sources</i>
- <ul><li> <del>Level detection</del>
- </li><li> <del>Frequency domain analysis</del>
- </li></ul>
- </li><li> <i>Synthesis of sources</i>
- <ul><li> <del>Generation of common signals for synthesis and parameter modulation purposes</del>
- </li><li> <del>The ability to read in standard definitions of wavetable instruments</del>
- </li><li> <del>Acceptable performance of synthesis</del>
- </li></ul>
- </li></ul>
+ <p>General consensus that while this is an interesting use case, there is no clamor to facilitate it entirely and urgently.</p>
+
<h4>UC5 — Other Requirements TBA? </h4>
<p>The context of a music writing application introduces some additional high level requirements on a sequencing/synthesis subsystem:
</p>
@@ -572,53 +532,65 @@
<h2>Requirements </h2>
<section>
<h3>Sources of audio </h3>
- <p>The Audio Processing API can operate on a number of sources of audio:
- </p>
- <ul><li> a DOM Element can be a source: HTML <audio> elements (with both remote and local sources)
- </li><li> memory-resident PCM data can be a source: Individual memory-resident “buffers” of PCM audio data which are not associated with <audio> elements
- </li><li> programmatically calculated data can be a source: on-the-fly generation of audio data
- </li><li> devices can act as a source: Audio that has been captured from devices - microphones, instruments etc.
- </li><li> remote peer can act as a source: Source from a remote peer (e.g. a WebRTC source)
- </li></ul>
+
+ <p>The Audio Processing API can operate on a number of sources of audio: </p>
+
+ <ul>
+ <li> a DOM Element can be a source: HTML <audio> elements (with both remote and local sources)</li>
+ <li> memory-resident PCM data can be a source: Individual memory-resident “buffers” of PCM audio data which are not associated with <audio> elements</li>
+ <li> programmatically calculated data can be a source: on-the-fly generation of audio data</li>
+ <li> devices can act as a source: Audio that has been captured from devices - microphones, instruments etc.</li>
+ <li> remote peer can act as a source: Source from a remote peer (e.g. a WebRTC source) </li>
+ </ul>
+
+
<h4>Support for primary audio file formats </h4>
- <p>Sources of audio can be compressed or uncompressed, in typical standard formats found on the Web and in the industry (e.g. MP3 or WAV)
- </p>
+ <p>Sources of audio can be compressed or uncompressed, in typical standard formats found on the Web and in the industry (e.g. MP3 or WAV)</p>
+
<h4>One source, many sounds </h4>
- <p>It should be possible to load a single source of sound and instantiate it in multiple, overlapping occurrences without reloading that are mixed together.
- </p>
+ <p>It should be possible to load a single source of sound and instantiate it in multiple, overlapping occurrences without reloading that are mixed together.</p>
+
<h4>Playing / Looping sources of audio </h4>
<p>A subrange of a source can be played. It should be possible to start and stop playing a source of audio at any desired offset within the source. This would then allow the source to be used as an audio sprite.
See: <a href="http://remysharp.com/2010/12/23/audio-sprites/" class="external free" title="http://remysharp.com/2010/12/23/audio-sprites/">http://remysharp.com/2010/12/23/audio-sprites/</a>
- And: <a href="http://lists.w3.org/Archives/Public/public-audio/2012JanMar/0006.html" class="external free" title="http://lists.w3.org/Archives/Public/public-audio/2012JanMar/0006.html">http://lists.w3.org/Archives/Public/public-audio/2012JanMar/0006.html</a>
- </p><p>A source can be looped. It should be possible to loop memory-resident sources. It should be possible to loop on a whole-source and intra-source basis, or to play the beginning of a sound leading into a looped segment.
- </p><p><i>Example:</i>
- </p>
- <pre>
- // load source
- source.loop = true;
- source.play();
-
- </pre>
- <p><br />
- </p>
+ And: <a href="http://lists.w3.org/Archives/Public/public-audio/2012JanMar/0006.html" class="external free" title="http://lists.w3.org/Archives/Public/public-audio/2012JanMar/0006.html">http://lists.w3.org/Archives/Public/public-audio/2012JanMar/0006.html</a></p>
+ <p>A source can be looped. It should be possible to loop memory-resident sources. It should be possible to loop on a whole-source and intra-source basis, or to play the beginning of a sound leading into a looped segment.</p>
+
<h4>Capture of audio from microphone, line in, other inputs </h4>
- <p>Audio from a variety of sources, including line in and microphone input, should be made available to the API for processing.
- </p>
+ <p>Audio from a variety of sources, including line in and microphone input, should be made available to the API for processing.</p>
+
<h4>Adding effects to the audio part of a video stream, and keep it in sync with the video playback </h4>
- <p>The API should have access to the audio part of video streams being played in the browser, and it should be possible to add effects to it and output the result in real time during video playback, keeping audio and video in sync.
- </p>
+ <p>The API should have access to the audio part of video streams being played in the browser, and it should be possible to add effects to it and output the result in real time during video playback, keeping audio and video in sync.</p>
+
<h4>Sample-accurate scheduling of playback </h4>
- <p>In the case of memory-resident sources it should be possible to trigger the playback of the audio in a sample-accurate fashion.
- </p>
+ <p>In the case of memory-resident sources it should be possible to trigger the playback of the audio in a sample-accurate fashion. </p>
+
+
<h4>Buffering </h4>
- <p>From: <a href="http://lists.w3.org/Archives/Public/public-audio/2012JanMar/0006.html" class="external free" title="http://lists.w3.org/Archives/Public/public-audio/2012JanMar/0006.html">http://lists.w3.org/Archives/Public/public-audio/2012JanMar/0006.html</a>
- </p>
- <dl><dd>It would be nice to have something like AudioNode.onready(123.2, callback)
- </dd><dd>if the browser is really sure to playback properly.
- </dd></dl>
+ <p>From: <a href="http://lists.w3.org/Archives/Public/public-audio/2012JanMar/0006.html" class="external free" title="http://lists.w3.org/Archives/Public/public-audio/2012JanMar/0006.html">http://lists.w3.org/Archives/Public/public-audio/2012JanMar/0006.html</a></p>
+ <dl>
+ <dd>It would be nice to have something like AudioNode.onready(123.2, callback)</dd>
+ <dd>if the browser is really sure to playback properly.</dd>
+ </dl>
+
+
<h4>Support for basic polyphony </h4>
- <p>A large number of simultaneous sources must be able to be played back simultaneously. As a guideline, the use-cases have identified that 32 [TODO: validate the number based on FMOD] simultaneous audio sources are required for typical music and gaming applications.
- </p>
+
+ <p>A large number of simultaneous sources must be able to be played back simultaneously. As a guideline, the use-cases have identified that 32 [TODO: validate the number based on FMOD] simultaneous audio sources are required for typical music and gaming applications.</p>
+
+ <h4>Multi-channel support</h4>
+ <p>The API should support multi-channel (surround) sounds, in particular:</p>
+ <ul>
+ <li>Channel layouts and mapping should be supported. Mapping of typical layouts (mono, stereo, quad, 5.1 and 7.1) should be clearly specified.</li>
+ <li>Channel up and down-mixing should be allowed. That means
+ <ul>
+ <li>The ability to match the number of channels supported by the hardware (with no upper limit?)</li>
+ <li>The ability to up mix any source to match the number of channels of the system</li>
+ <li>The ability to use as source an audio stream with more channels than the system supports</li>
+ <li>The ability to down mix a source or stream to any number of channels, down to mono</li>
+ </ul>
+ </li>
+ </ul>
<h4>Rapid scheduling of many independent sources </h4>
<p>The ability to construct and schedule the playback of approximately 100 notes or sonic events per second across all voices would be required for typical music synthesis and gaming applications.
</p>
@@ -649,16 +621,16 @@
<p>Where there are parameters for these effects, it should be possible to automatically modify these parameters in a programmatic, time-dependent way. Parameter changes must be able to be scheduled relative to a source’s onset time which may be in the future. Primary candidates for automation include gain, playback rate and filter frequency.
</p><p>Transformations include:
</p>
- <h4>gain adjustment </h4>
- <h4>playback rate adjustment </h4>
- <h4>spatialization </h4>
+ <h4>Gain adjustment </h4>
+ <h4>Playback rate adjustment </h4>
+ <h4>Spatialization </h4>
<ul><li> equal power/level panning
</li><li> binaural HRTF-based spatialization
</li><li> including the influence of the directivity of acoustic sources
</li><li> including the attenuation of acoustic sources by distance
</li><li> including the effect of movement on acoustic sources
</li></ul>
- <h4>filtering </h4>
+ <h4>Filtering </h4>
<ul><li> graphic EQ
</li><li> low/hi/bandpass filters
</li><li> impulse response filters
@@ -668,7 +640,7 @@
<h4>Noise gating </h4>
<p>It is possible to apply a real-time noise gate to a source to automatically mute it when its average power level falls below some arbitrary threshold (is this a reflexive case of ducking?).
</p>
- <h4>dynamic range compression </h4>
+ <h4>Dynamic range compression </h4>
<p><b>TBA</b>
</p>
<h4>The simulation of acoustic spaces </h4>
@@ -731,7 +703,7 @@
<th style="width: 20em">Requirement Priority</th>
<th>UC 1: Video Chat</th>
<th>UC 2: HTML5 game with audio effects, music</th>
- <th>UC 3: online music production tool</th>
+ <th>UC 3: Online music production tool</th>
<th>UC 4: Online radio broadcast</th>
<th>UC 5: writing music on the web</th>
<th>UC 6: wavetable synthesis of a virtual music instrument</th>
@@ -803,11 +775,17 @@
<td> </td> <td>✓</td> <td> </td> <td>✓</td> <td> </td> <td> </td> <td>✓</td> <td> </td> <td>✓</td> <td> </td> <td> </td> <td> </td> <td>✓</td> <td> </td> <td> </td>
</tr>
<tr>
+ <td> Multi-channel support</td>
+ <td>Baseline</td>
+ <td>✓</td> <td>✓</td> <td>✓</td> <td>✓</td> <td>✓</td> <td>✓</td> <td>✓</td> <td>✓</td> <td>✓</td> <td>✓</td> <td>✓</td> <td>✓</td> <td>✓</td> <td>✓</td> <td>✓</td>
+ </tr>
+ <tr>
<td> Support for basic polyphony </td>
<td>Baseline</td>
<td>✓</td> <td>✓</td> <td>✓</td> <td>✓</td> <td>✓</td> <td>✓</td> <td> </td> <td>✓</td> <td> </td> <td> </td> <td>✓</td> <td>✓</td> <td> </td> <td> </td> <td> </td>
</tr>
<tr>
+
<td> Rapid scheduling of many independent sources </td>
<td>Minority, but important</td>
<td> </td> <td>✓</td> <td>✓</td> <td> </td> <td>✓</td> <td>✓</td> <td> </td> <td> </td> <td> </td> <td> </td> <td> </td> <td>✓</td> <td> </td> <td> </td> <td> </td>