--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/media-stream-capture/ImageCapture.html Wed May 01 16:43:32 2013 -0700
@@ -0,0 +1,403 @@
+<!DOCTYPE html>
+<html xmlns='http://www.w3.org/1999/xhtml' lang='en'>
+ <head>
+ <meta charset='utf-8'/>
+ <title>Mediastream Image Capture</title>
+ <script class='remove'>
+ var respecConfig = {
+ specStatus: "ED"
+ , shortName: "image-capture"
+ , editors: [
+ { name: "Giridhar Mandyam"
+ , company: "Qualcomm Innovation Center, Inc"
+ , companyURL: "http://www.qualcomm.com/about/businesses/quicinc" }
+ ]
+ , edDraftURI: "http://gmandyam.github.com/image-capture"
+ , previousURI: "http://gmandyam.github.com/image-capture"
+ , copyrightStart: 2012
+ , noIDLIn: true
+ , wg: "Media Capture Task Force"
+ , wgURI: "http://www.w3.org/wiki/Media_Capture"
+ , wgPublicList: "public-media-capture"
+ , wgPatentURI: "XXX"
+ , isRecTrack: false
+ , isNoTrack: true
+ , format: 'markdown'
+ };
+ </script>
+ <script src='https://www.w3.org/Tools/respec/respec-w3c-common.js' class='remove' async></script>
+ </head>
+ <body>
+ <section id='sotd'>
+ Comments on this document are welcomed.
+ </section>
+
+ <section id='abstract'>
+ This document specific the takePhoto() and getFrame methods, and corresponding camera settings for use with MediaStreams as defined in Media Capture and Streams [[!GETUSERMEDIA]].
+ </section>
+
+ Introduction
+ ------------
+ <p>The API defined in this document taks a valid MediaStream and returns an encoded image in the form of a <code>Blob</code> (as defined in [[!FILE-API]]). The image is
+ provided by the capture device that provides the MediaStream. Moreover,
+ picture-specific settings can be optionally provided as arguments that can be applied to the image being captured.</p>
+
+ Image Capture API
+ --------------
+ <dl title='[Constructor(MediaStreamTrack track)] interface ImageCapture : EventTarget' class='idl'>
+ <dd>interface ImageCapture: EventTarget</dd>
+ <dt>readonly attribute PhotoSettingsOptions photoSettingsOptions</dt>
+ <dd>Describes current photo settings</dd>
+ <dt>readonly attribute MediaStreamTrack videoStreamTrack</dt>
+ <dd>The MediaStream passed into the constructor</dd>
+ <dt>attribute EventHandler onphoto</dt>
+ <dd>Register/unregister for photo events of type <code>BlobEvent</code>. The handler should expect to get a <code>BlobEvent</code> object as its first parameter.</dd>
+ <dt>attribute EventHandler onerror</dt>
+ <dd>Register/unregister for Image Capture error events of type <code>ImageCaptureErrorEvent</code>. The handler should expect to get a <code>ImageCaptureError</code> object as its first parameter.</dd>
+ <dt>attribute EventHandler onphotosettingschange</dt>
+ <dd>Register/unregister for photo settings change events of type <code>SettingsChangeEvent</code>.</dd>
+ <dt>attribute EventHandler onframegrab</dt>
+ <dd>Register/unregister for frame capture events of type <code>FrameGrabEvent</code>. The handler should expect to get a <code>FrameGrabEvent</code> object as its first parameter.</dd>
+ <dt>void setOptions(PhotoSettings? photoSettings)</dt>
+ <dd>When the <code>setOptions()</code> method of an <code>ImageCapture</code> object is invoked, then
+ then a valid <code>PhotoSettings</code> object <em title="must" class="rfc2119">must</em> be passed in the method to the
+ <code>ImageCapture</code> object. If the UA can successfully apply the settings, then the UA <em title="must" class="rfc2119">must</em> fire a <code>SettingsChangeEvent</code> event at the
+ <code>onphotosettingschange</code> event handler (if specified). If the UA cannot successfully apply the settings, then the UA
+ <em title="must" class="rfc2119">must</em> fire a <code>ImageCaptureErrorEvent</code> at the <code>ImageCapture</code> object whose <code>code</code> is set to SETTINGS_ERROR. </dd>
+ <dt>void takePhoto ()</dt>
+ <dd>When the <code>takePhoto()</code> method of an <code>ImageCapture</code> object is invoked,
+ then if the <code>MediaStreamTrack</code> provided in the contructor does not have the
+ <code>kind</code> attribute set to "video" or if the <code>MediaStreamTrack</code>'s <code>readyState</code> is not "live", the UA <em title="must" class="rfc2119">must</em> fire a <code>ImageCaptureErrorEvent</code> event at the <code>ImageCapture</code> object with a
+ new <code>ImageCaptureError</code> object whose <code>code</code> is set to INVALID_TRACK. If the UA is unable to execute the <code>takePhoto()</code> method for any
+ other reason (for example, upon invocation of multiple takePhoto() method calls in rapid succession), then the UA <em title="must" class="rfc2119">must</em> fire a <code>ImageCaptureErrorEvent</code> event at the <code>ImageCapture</code> object with a
+ new <code>ImageCaptureError</code> object whose <code>code</code> is set to PHOTO_ERROR.
+ Otherwise it <em title="must" class="rfc2119">must</em>
+ queue a task, using the DOM manipulation task source, that runs the following steps:
+ <ol>
+ <li>Gather data from the <code>MediaStreamTrack</code> into a <code>Blob</code> containing a single still image. The method of doing
+ this will depend on the underlying device. Devices
+ may temporarily stop streaming data, reconfigure themselves with the appropriate photo settings, take the photo,
+ and then resume streaming. In this case, the stopping and restarting of streaming <em title="should" class="rfc2119">should</em>
+ cause <code>mute</code> and <code>unmute</code> events to fire on the Track in question. </li>
+ <li>Raise a <code>BlobEvent</code> event containing the <code>Blob</code> to the <code>onphoto</code> event handler (if specified).</li></dd>
+ <dt>void getFrame()</dt>
+ <dd>When the <code>getFrame()</code> method of an <code>ImageCapture</code> object is invoked, then if the <code>MediaStreamTrack</code> provided in the contructor does not have the
+ <code>kind</code> attribute set to "video" or if the <code>MediaStreamTrack</code>'s <code>readyState</code> is not "live", the UA <em title="must" class="rfc2119">must</em> fire a <code>ImageCaptureErrorEvent</code> event at the <code>ImageCapture</code> object with a
+ new <code>ImageCaptureError</code> object whose <code>code</code> is set to INVALID_TRACK. If the UA is unable to execute the <code>getFrame()</code> method for any
+ other reason, then the UA <em title="must" class="rfc2119">must</em> fire a <code>ImageCaptureErrorEvent</code> event at the <code>ImageCapture</code> object with a
+ new <code>ImageCaptureError</code> object whose <code>code</code> is set to FRAME_GRAB_ERROR. Otherwise it <em title="must" class="rfc2119">must</em>
+ queue a task, using the DOM manipulation task source, that runs the following steps:
+ <ol>
+ <li>Gather data from the <code>MediaStreamTrack</code> into a <code>ImageData</code> object (as defined in [[!CANVAS-2D]]) containing a single still frame in RGBA format. The <code>width</code> and <code>height</code> of the
+ <code>ImageData</code> object are derived from the constraints of the <code>MediaStreamTrack</code>. </li>
+ <li>Raise a <code>FrameGrabEvent</code> event containing the <code>ImageData</code> to the <code>onframegrab</code> event handler (if specified). {Note: <code>getFrame()</code> returns data only once upon being invoked.}</li>
+ </dd>
+ </dl>
+
+ <code>FrameGrabEvent</code>
+ --------------
+ <dl title='[Constructor(DOMString type, optional FrameGrabEventInit frameGrabInitDict)] interface FrameGrabEvent : Event' class='idl'>
+ <dt>readonly attribute ImageData imageData</dt>
+ <dd>Returns an <code>ImageData</code> object whose <code>length</code> and <code>height</code> attributes indicates the dimensions of the captured frame. </dd>
+ </dl>
+
+ ##### <code>FrameGrabEventInit</code> Dictionary
+
+ <dl title='dictionary FrameGrabEventInit : EventInit' class='idl'>
+ <dt>ImageData imageData</dt>
+ <dd>An <code>ImageData</code> object containing the data to deliver via this event.</dd>
+ </dl>
+
+ <code>ImageCaptureErrorEvent</code>
+ --------------
+ <dl title='[Constructor(DOMString type, optional ImageCaptureErrorEventInit imageCaptureErrorInitDict)] interface ImageCaptureErrorEvent : Event' class='idl'>
+ <dt>readonly attribute ImageCaptureError imageCaptureError</dt>
+ <dd>Returns a <code>ImageCaptureError</code> object whose <code>code</code> attribute indicates the type of error occurrence. </dd>
+ </dl>
+
+ ##### <code>ImageCaptureErrorEventInit</code> Dictionary
+
+ <dl title='dictionary ImageCaptureErrorEventInit : EventInit' class='idl'>
+ <dt>ImageCaptureError imageCaptureError</dt>
+ <dd>A <code>ImageCaptureError</code> object containing the data to deliver via this event.</dd>
+ </dl>
+
+ <code>BlobEvent</code>
+ --------------
+ <dl title='[Constructor(DOMString type, optional BlobEventInit blobInitDict)] interface BlobEvent : Event' class='idl'>
+ <dt>readonly attribute Blob data</dt>
+ <dd>Returns a <code>Blob</code> object whose type attribute indicates the encoding of the blob data. An implementation must return a Blob in a format that is capable of being viewed in an HTML <code><img></code> tag. </dd>
+ </dl>
+
+ ##### <code>BlobEventInit</code> Dictionary
+
+ <dl title='dictionary BlobEventInit : EventInit' class='idl'>
+ <dt>Blob data</dt>
+ <dd>A <code>Blob</code> object containing the data to deliver via this event.</dd>
+ </dl>
+
+ <code>SettingsChangeEvent</code>
+ --------------
+ <dl title='[Constructor(DOMString type, optional SettingsChangeEventInit photoSettingsInitDict)] interface PhotoSettingsEvent : Event' class='idl'>
+ <dt>readonly attribute PhotoSettings photoSettings</dt>
+ <dd>Returns a <code>PhotoSettings</code> object whose type attribute indicates the current photo settings. </dd>
+ </dl>
+
+ ##### <code>SettingsChangeEventInit</code> Dictionary
+
+ <dl title='dictionary SettingsChangeEventInit : EventInit' class='idl'>
+ <dt>PhotoSettings photoSettings</dt>
+ <dd>A <code>PhotoSettings</code> object containing the data to deliver via this event.</dd>
+ </dl>
+
+ <code>ImageCaptureError</code>
+ -----------------
+ <p>The <code>ImageCaptureError</code> object is passed to an <code>onerror</code> event handler of an
+ <code>ImageCapture</code> object if an error occurred when the object was created or any of its methods were invoked.</p>
+ <dl title='[NoInterfaceObject] interface ImageCaptureError' class='idl'>
+ <dt>const unsigned short FRAME_GRAB_ERROR=1</dt>
+ <dd>An <code>ImageCaptureError</code> object must set its <code>code</code> value to this constant if an error occurred upon invocation of the <code>getFrame()</code> method of the <code>ImageCapture</code> interface.</dd>
+ <dt>const unsigned short SETTINGS_ERROR=2</dt>
+ <dd>An <code>ImageCaptureError</code> object must set its <code>code</code> value to this constant if an error occurred upon invocation of the <code>setOptions()</code> method of the <code>ImageCapture</code> interface.</dd>
+ <dt>const unsigned short PHOTO_ERROR=3</dt>
+ <dd>An <code>ImageCaptureError</code> object must set its <code>code</code> value to this constant if an error occurred upon invocation of the <code>takePhoto()</code> method of the <code>ImageCapture</code> interface.</dd>
+ <dt>const unsigned short ERROR_UNKNOWN=4</dt>
+ <dd>An <code>ImageCaptureError</code> object must set its <code>code</code> value to this constant if an error occurred due to indeterminate cause upon invocation of any method of the <code>ImageCapture</code> interface.</dd>
+ <dt>readonly attribute unsigned short code</dt>
+ <dd>The <code>code</code> attribute returns the appropriate code for the error event, derived from the constants defined in the <code>ImageCaptureError</code> interface.</dd>
+ <dt>readonly attribute DOMString message</dt>
+ <dd>The <code>message</code> attribute must return an error message describing the details of the error encountered.</dd>
+ </dl>
+
+ <section>
+ <h2><code>MediaSettingsRange</code></h2>
+ <dl title='interface MediaSettingsRange' class='idl'>
+ <dt>readonly attribute unsigned long max</dt>
+ <dd>The maximum value of this setting</dd>
+ <dt>readonly attribute unsigned long min</dt>
+ <dd>The minimum value of this setting</dd>
+ <dt>readonly attribute unsigned long initial</dt>
+ <dd>The current value of this setting</dd>
+ </dl>
+ </section>
+
+ <section>
+ <h2><code>MediaSettingsItem</code></h2>
+ <p>The <code>MediaSettingsItem</code> interface is now defined, which allows for a single setting to be managed.</p>
+ <dl title='interface MediaSettingsItem' class='idl'>
+ <dt>readonly attribute any value</dt>
+ <dd>Value of current setting.</dd>
+ </dl>
+ <p></p>
+ </section>
+
+ <section>
+ <h2><code>PhotoSettingsOptions</code></h2>
+ <p>The PhotoSettingsOptions attribute of the <code>ImageCapture</code> object provides
+ the photo-specific settings options and current settings values. The following definitions are assumed
+ for individual settings and are provided for information purposes:</p>
+ <ol>
+ <li><i>White balance mode</i> is a setting that cameras use to adjust for different color temperatures. Color temperature is
+ the temperature of background light (measured in Kelvin normally). This setting can also be automatically
+ determined by the implementation. If 'automatic' mode is selected, then the Kelvin setting for White Balance Mode
+ may be overridden. Typical temprature ranges for different modes are provided below:
+ <table border="1">
+ <tr>
+ <th>Mode</th>
+ <th>Kelvin range</th>
+ </tr>
+ <tr>
+ <td>incandescent</td>
+ <td>2500-3500</td>
+ </tr>
+ <tr>
+ <td>fluorescent</td>
+ <td>4000-5000</td>
+ </tr>
+ <tr>
+ <td>warm-fluorescent</td>
+ <td>5000-5500</td>
+ </tr>
+ <tr>
+ <td>daylight</td>
+ <td>5500-6500</td>
+ </tr>
+ <tr>
+ <td>cloudy-daylight</td>
+ <td>6500-8000</td>
+ </tr>
+ <tr>
+ <td>twilight</td>
+ <td>8000-9000</td>
+ </tr>
+ <tr>
+ <td>shade</td>
+ <td>9000-10000</td>
+ </tr>
+ </table>
+ </li>
+ <li><i>Exposure</i> is the amount of light allowed to fall on the photographic medium. Auto-exposure mode is a camera setting
+ where the exposure levels are automatically adjusted by the implementation based on the subject of the photo.</li>
+ <li><i>Exposure Compensation</i> is a numeric camera setting that adjusts the exposure level from the current value used by the implementation. This value can
+ be used to bias the exposure level enabled by auto-exposure.</li>
+ <li>The <i>ISO</i> setting of a camera describes the sensistivity of the camera to light. It is a numeric value, where the lower the value
+ the greater the sensitivity. This setting in most implementations relates to shutter speed, and is sometimes known as the ASA setting.</li>
+ <li><i>Red Eye Reduction</i> is a feature in cameras that is designed to limit or prevent the appearance of
+ red pupils ("Red Eye") in photography subjects due prolonged exposure to a camera's flash.</li>
+ <li><i>Brightness</i> refers to the numeric camera setting that adjusts the perceived amount of light emitting from the photo object. A higher brightness setting increases the intensity of darker areas in a scene while compressing the intensity of brighter parts of the scene.</li>
+ <li><i>Contrast</i> is the numeric camera setting that controls the difference in brightness between light and dark areas in a scene. A higher contrast setting reflects an expansion in the difference in brightness.</li>
+ <li><i>Saturation</i> is a numeric camera setting that controls the intensity of color in a scene (i.e. the amount of gray in the scene). Very low saturation levels will result in photo's closer to black-and-white.</li>
+ <li><i>Sharpness</i> is a numeric camera setting that controls the intensity of edges in a scene. Higher sharpness settings result in higher edge intensity, while lower settings result in less contrast and blurrier edges (i.e. soft focus).</li>
+ </ol>
+ <dl title='interface PhotoSettingsOptions' class='idl'>
+ <dt>attribute MediaSettingsItem autoWhiteBalanceMode</dt>
+ <dd>This reflects whether automated White Balance Mode selection is on or off, and is boolean - on is true</dd>
+ <dt>attribute MediaSettingsRange whiteBalanceMode</dt>
+ <dd>This reflects the current white balance mode setting. Values are of type <code>WhiteBalanceModeEnum</code>.</dd>
+ <dt>attribute ExposureMode autoExposureMode</dt>
+ <dd>This reflects the current auto exposure mode setting. Values are of type <code>ExposureMode</code>.</dd>
+ <dt>attribute MediaSettingsRange exposureCompensation</dt>
+ <dd>This reflects the current exposure compensation setting and permitted range. Values are numeric.</dd>
+ <dt>attribute MediaSettingsRange iso</dt>
+ <dd>This reflects the current camera ISO setting and permitted range. Values are numeric.</dd>
+ <dd>This feature reflects the current exposure level for recorded images. Values are numeric.</dd>
+ <dt>attribute MediaSettingsItem redEyeReduction</dt>
+ <dd>This reflects whether camera red eye reduction is on or off, and is boolean - on is true</dd>
+ <dt>attribute MediaSettingsRange brightness</dt>
+ <dd>This reflects the current brightness setting of the camera and permitted range. Values are numeric.</dd>
+ <dt>attribute MediaSettingsRange constrast</dt>
+ <dd>This reflects the current contrast setting of the camera and permitted range. Values are numeric.</dd>
+ <dt>attribute MediaSettingsRange saturation</dt>
+ <dd>This reflects the current saturation setting of the camera and permitted range. Values are numeric.</dd>
+ <dt>attribute MediaSettingsRange sharpness</dt>
+ <dd>This reflects the current sharpness setting of the camera and permitted range. Values are numeric.</dd>
+ <dt>attribute MediaSettingsRange imageHeight</dt>
+ <dd>This reflects the image height range supported by the UA and the current height setting.</dd>
+ <dt>attribute MediaSettingsRange imageWidth</dt>
+ <dd>This reflects the image width range supported by the UA and the current width setting.</dd>
+ </dl>
+ <p></p>
+ </section>
+
+ <section>
+ <h2><code>ExposureMode</code></h2>
+ <dl title='enum ExposureModeEnum' class='idl'>
+ <dt>frame-average</dt>
+ <dd>Average of light information from entire scene</dd>
+ <dt>center-weighted</dt>
+ <dd>Sensitivity concentrated towards center of viewfinder</dd>
+ <dt>spot-metering</dt>
+ <dd>Spot-centered weighting</dd>
+ </dl>
+ </section>
+
+ <section>
+ <h2><code>PhotoSettings</code></h2>
+ <p>The <code>PhotoSettings</code> object is optionally passed into the <code>ImageCapture.setOptions()</code> method
+ in order to modify capture device settings specific to still imagery. Each of the attributes in this object
+ are optional.</p>
+ <dl title='dictionary PhotoSettings' class='idl'>
+ <dt>attribute boolean autoWhiteBalanceMode</dt>
+ <dd>This reflects whether automatic White Balance Mode selection is desired.</dd>
+ <dt>attribute unsigned long whiteBalanceMode</dt>
+ <dd>This reflects the desired white balance mode setting.</dd>
+ <dt>attribute any autoExposureMode</dt>
+ <dd>This reflects the desired auto exposure mode setting. Acceptable values are of type <code>ExposureModeEnum</code>.</dd>
+ <dt>attribute unsigned long exposureCompensation</dt>
+ <dd>This reflects the desired exposure compensation setting.</dd>
+ <dt>attribute unsigned long iso</dt>
+ <dd>This reflects the desired camera ISO setting.</dd>
+ <dd>This feature reflects the current exposure level for recorded images. Values are numeric.</dd>
+ <dt>attribute boolean redEyeReduction</dt>
+ <dd>This reflects whether camera red eye reduction is desired</dd>
+ <dt>attribute unsigned long brightness</dt>
+ <dd>This reflects the desired brightness setting of the camera.</dd>
+ <dt>attribute unsigned long constrast</dt>
+ <dd>This reflects the desired contrast setting of the camera.</dd>
+ <dt>attribute unsigned long saturation</dt>
+ <dd>This reflects the desired saturation setting of the camera.</dd>
+ <dt>attribute unsigned long sharpness</dt>
+ <dd>This reflects the desired sharpness setting of the camera.</dd>
+ <dt>attribute unsigned long imageHeight</dt>
+ <dd>This reflects the desired image height. The UA <em title="must" class="rfc2119">must</em> select the closest height value this setting if it supports a discrete set of height options. </dd>
+ <dt>attribute unsigned long imageWidth</dt>
+ <dd>This reflects the desired image width. The UA <em title="must" class="rfc2119">must</em> select the closest width value this setting if it supports a discrete set of width options.</dd>
+ </dl>
+ </section>
+
+ Examples
+ -------
+ ##### Taking a picture if Red Eye Reduction is activated
+ <pre class='example'>
+ navigator.getUserMedia({video: true}, gotMedia, failedToGetMedia);
+
+ function gotMedia(mediastream) {
+ //Extract video track. 'kind' attribute not checked because stream was created with video option only.
+ var videoDevice = mediastream.getTrackByID()[0];
+ // Check if this device supports a picture mode...
+ var pictureDevice = new ImageCapture(videoDevice);
+ if (pictureDevice) {
+ pictureDevice.onphoto = showPicture;
+ if (pictureDevice.photoSettingsOptions.redEyeReduction) {
+ pictureDevice.setOptions({redEyeReductionSetting:true});
+ }
+ else
+ console.log('No red eye reduction');
+ pictureDevice.onphotosettingschange = function(){
+ if (pictureDevice.photoSettingsOptions.redEyeReduction.value)
+ pictureDevice.takePhoto();
+ }
+ }
+ }
+
+ function showPicture(e) {
+ var img = document.querySelector("img");
+ img.src = URL.createObjectURL(e.data);
+ }
+
+ function failedToGetMedia{
+ console.log('Stream failure');
+ }
+ </pre>
+
+ ##### Grabbing a Frame for Post-Processing
+ <pre class='example'>
+ navigator.getUserMedia({video: true}, gotMedia, failedToGetMedia);
+
+ function gotMedia(mediastream) {
+ //Extract video track. 'kind' attribute not checked because stream was created with video option only.
+ var videoDevice = mediastream.getTrackByID()[0];
+ // Check if this device supports a picture mode...
+ var pictureDevice = new ImageCapture(videoDevice);
+ if (pictureDevice) {
+ pictureDevice.onframegrab = processFrame;
+ pictureDevice.getFrame();
+ }
+ }
+
+ function processFrame(e) {
+ imgData = e.imageData;
+ width = imgData.width;
+ height = imgData.height;
+ for (j=3; j < imgData.length; j+=4)
+ {
+ // Set all alpha values to medium opacity
+ imgData.data[j] = 128;
+ }
+ // Create new ImageObject with the modified pixel values
+ var canvas = document.createElement('canvas');
+ ctx = canvas.getContext("2d");
+ newImg = ctx.createImageData(width,height);
+ for (j=0; j < imgData.length; j++)
+ {
+ newImg.data[j] = imgData.data[j];
+ }
+ // ... and do something with the modified image ...
+ }
+
+ function failedToGetMedia{
+ console.log('Stream failure');
+ }
+ </pre>
+ </body>
+</html>