This proposal describes additions and suggested changes to the Media Capture and Streams specification in order to support the goal of device settings retrieval and modification. This proposal incorporates feedback from three prior proposals with the same goal [v3] [v2] [v1].

Remove LocalMediaStream interface

In this proposal, the derived LocalMediaStream interface is removed. Rather than returning a LocalMediaStream instance in the NavigatorUserMediaSuccessCallback, a vanilla MediaStream object is returned. The primary difference is in the tracks contained in that MediaStream object.

Rationale

The LocalMediaStream object currently extends MediaStream by adding a single method "stop()". In my prior proposals, this object was radically altered in order to facilitate several goals:
Provide a predictable home for developers to find and modify device settings
A previous proposal went out of its way to strongly associate LocalMediaStream objects with devices. This seemed like a good design because local device configuration is always on the local media stream. This made for a stable, dependable API surface for all local media stream instances (no guesswork).
Prevent track-list mutations
A previous proposal also removed the track lists on local media streams (resulting in some dramatic inheritance changes). Mutable tracks lists on LocalMediaStream objects seemed like the wrong design considering the current thinking that a getUserMedia request would only ever produce a LocalMediaStream with at most one audio or video track.

Some feedback even suggested re-considering the "at most one video/audio track per request to getUserMedia".

While thinking about these goals and the feedback, I began to consider a few things:

Device-centric tracks
With tracks supplemented with device-characteristics (duck-typing), the LocalMediaStream's stop() API was a convenience feature for stopping all tracks backed by a device on the LocalMediaStream object. With device- centric tracks a stop() API should be present on the tracks themselves.
Mutable track lists
Mutable track lists were not a desirable feature while I was locked into considering the LocalMediaStream as strongly associated with device-control. What is actually necessary, is that there is a something immutable associated with devices--that "something" doesn't necessarily need to be a LocalMediaStream or any MediaStream-like object at all! Once I unlocked that line of thinking, I began to experiment with the notion of a device list which then ultimately brought back a use-case for having mutable track lists for MediaStream objects. (It did not bring back a need for LocalMediaStream objects themselves though.)
Workflow for access to additional device streams
It is now understood that to request additional streams for different devices (e.g., the second camera on a dual-camera mobile phone), one must invoke getUserMedia a second time. In my prior proposal, this would result in a separate LocalMediaStream instance. At this point there are two LocalMediaStream objects each with their own devices. While this was nice for consistency of process, it was a challenge in terms of use of the objects with a MediaStream consumer like the Video tag.

To illustrate this challenge, consider how my prior proposal required a re-hookup of the MediaStream to a video tag consumer:

  1. First request to getUserMedia
  2. LocalMediaStream (1) obtained from success callback
  3. createObjectURL and preview in a video tag
  4. Second call to getUserMedia
  5. LocalMediaStream (2) obtained from success callback
  6. createObjectURL and preview in same video tag

Note that this process has to bind a completely new LocalMediaStream to the video tag a second time (if re-using the same video tag) only because the second LocalMediaStream object was different than the first.

It is much more efficient for developer code to simply add/remove tracks to a MediaStream that are relevant, without needing to change the consumer of the MediaStream.

Usage of getUserMedia for permission rather than for additional device access
The getUserMedia method is the gateway for permission to media. This proposal does not suggest changing that concept. It does suggest, however, that more information can be made available for discovery of additional devices within the approved "category" or "type" of media, and provide a way to obtain those additional devices without needing to go through the "permissions" route (i.e., getUserMedia).
Importance of restricting control to LocalMediaStream
Upon reflection of the feedback around the prior proposal, the relative importance of restricting control of the devices associated with tracks on the LocalMediaStream to only the LocalMediaStream did not seem as vital, insofar as the device-level access via the track is not directly available through a PeerConnection to a remote browser.

New MediaStreamTrack (derived) types

This proposal consolidates settings directly into the tracks that are provided by devices. However, in order to do this efficiently and in a future-extensible manner, the highly-generic MediaStreamTrack is now extended for specific characteristics of the devices it embodies, resulting in a hierarchy:

Local and remote video tracks

MediaStreamTrack objects that are of kind "video" and that are located in a MediaStream's videoTracks list will be instances of a VideoStreamTrack. The VideoStreamTrack provides basic (read-only) properties pertinent to all sources of video.

There is no takePicture API on a VideoStreamTrack because a simple frame-grab can be accomplished using a combination of a <video> and <canvas> APIs (takePicture is intended for use with a camera's high-resolution picture mode, not for arbitrary video frame capture).

I'm intentionally keeping this interface as sparse as possible. Features about the video that can be calculated like aspect ratio are not provided.

VideoStreamTrack interface

readonly attribute unsigned long width
The "natural" width (in pixels) of the video flowing through the track. In the case of a VideoDeviceTrack, this value represents the current setting of the camera's sensor (still in terms of number of pixels). This value is independent of the camera's rotation (if the camera's rotation setting is changed, it does not impact this value). For example, consider a camera setting with width of 1024 pixels and height of 768 pixels. If the camera's rotation setting is changed by 90 degrees, the width is still reported as 1024 pixels. However, a <video> element used to preview this track would report a width of 768 pixels (the effective width with rotation factored in).
readonly attribute unsigned long height
The "natural" height (in pixels) of the video in this track. See the "width" attribute for additional info.

Local and remote audio tracks

MediaStreamTrack objects that are of kind "audio" and that are located in a MediaStream's audioTracks list will be instances of an AudioStreamTrack. The AudioStreamTrack provides basic (read-only) properties pertinent to all sources of audio.

AudioStreamTrack interface

Camera device tracks

VideoDeviceTracks are created by the user agent to represent a camera device that provides local video.

VideoDeviceTrack interface

readonly attribute PictureDeviceTrack? pictureTrack
If the device providing this VideoDeviceTrack supports a "high-resolution picture mode", this attribute will be reference to a PictureDeviceTrack object. Otherwise, this attribute will be null.
readonly attribute VideoFacingEnum facing
From the user's perspective, this attribute describes whether this camera is pointed toward the user ("user") or away from the user ("environment"). If this information cannot be reliably obtained, for example from a USB external camera, the value "unknown" is returned.
void stop()
Causes this track to enter the ENDED state. Same behavior of the old LocalMediaStream's stop API, but only affects this device track.

VideoFacingEnum enumeration

unknown
The relative directionality of the camera cannot be determined by the user agent based on the hardware.
user
The camera is facing toward the user (a self-view camera).
environment
The camera is facing away from the user (viewing the environment).

Cameras with "high-resolution picture" modes

The PictureDeviceTrack interface is created by the user agent if the camera device providing the VideoDeviceTrack supports an optional "high-resolution picture mode" with picture settings different (better) from those of its basic video constraints.

This track is initially available from a VideoDeviceTrack via the pictureTrack property. This track type is not present in the video device list (MediaDeviceList). Likewise, it cannot be stopped directly, and its VideoStreamTrack inherited attributes reflect the values of its "owning" VideoDeviceTrack.

The PictureDeviceTrack is essentially a specialized VideoStreamTrack (this track type is of kind "video"). It may be explicitly added to a videoTracks list (MediaStreamTrackList) in order to output its track video to a <video> tag, but its preview video stream reflects the owning VideoDeviceTrack's settings, rather than the settings directly available on this object. Rather the settings of this object are only applied at the time when the takePicture API is invoked.

PictureDeviceTrack interface

void takePicture()
Temporarily mutes the owning VideoDeviceTrack's stream, then asynchronously switches the camera into "high resolution picture mode", applies the PictureDeviceTrack settings (a snapshot from the time the takePicture API was called, and records/encodes an image using a user-agent determined format into a Blob object. Finally, queues a task to fire a "picture" event with the resulting Blob instance.

Could consider providing a hint or setting for the desired picture format.

attribute EventHandler onpicture
Register/unregister for "picture" events. The handler should expect to get a PictureEvent object as its first parameter.

Is an "error" event necessary here too?

In the previous proposal, the PictureEvent returned a Canvas ImageData object, however it makes sense to return a compressed format (PNG/JPEG), especially given that picture snapshots will be very high resolution, and ImageData objects are essentially raw images.

PictureEvent interface

readonly attribute Blob data
Returns a Blob object whose type attribute indicates the encoding of the picture data. An implementation must return a Blob in a format that is capable of being viewed in an HTML <img> tag.

Microphone device tracks

AudioDeviceTracks are created by the user agent to represent a microphone device that provides local audio.

AudioDeviceTrack interface

readonly attribute unsigned long level
The sensitivity of the microphone. This value must be a whole number between 0 and 100 inclusive. When a MediaStreamTrack is muted, the level attribute must return 0. A value of 100 means the microphone is configured for maximum gain.
void stop()
Causes this track to enter the ENDED state. Same behavior of the old LocalMediaStream's stop API, but only for this device track.

Settings Retrieval/Application

As noted in prior proposals, camera/microphone settings must be applied asynchronously to ensure that web applications can remain responsive for all device types that may not respond quickly to setting changes.

My prior proposals used a monolithic dictionary of settings for inspection and application. This proposal takes a different approach, considering the feedback for more-direct access to settings, expected patterns for settings adjustment (which is generally one setting at at time as initiated by a web application UI), difficulties in understanding what values were read-only vs. writable, and the current already-defined constraint application engine.

Grouping setting features

Settings are organized into two groups: value ranges (a continuum of values) and enumerated values. Value ranges include a min and max value, while enumerated values are provided in an array with an associated length. Both groups of settings include an "initial" value, which is the value that is expected to be the device's default value when it is acquired.

The key to changing settings in either setting group is the request() API. This is the mechanism for asynchronously requesting that the device change the value of the setting that the setting group is applicable to. The mechanics for applying setting change requests follows exactly the model used when applying constraints at getUserMedia invocation. Each team a request() is made, the user agent begins building up an [internally-represented] constraint structure which is associated with the device making the request (and only that device). For example, if a "width" setting change request is made, the user agent creates a constraint structure equivalent to the following getUserMedia constraint (except that this constraint only applies to the specific device--not all video devices):

{ video: { optional: [ { width: value } ] } }

If this is the only request during this script-execution task, then when control returns to the user agent, this constraint will be committed (i.e., like an indexedDB transaction) and the constraint application logic will evaluate the request making changes to the current device if applicable.

If there is another request during the same script-execution task, it is appended to the optional list. Since order is important in the optional constraints list, the first requested setting has priority over the next.

The request() API also has a flag used to signal to the UA that the requested setting change should be mandatory. In this case, the constraint is added to the mandatory set, and replaces an existing setting in that set if the names collide (last setting wins). My expectation is that if a mandatory constraint cannot be satisfied, then the UA must end that stream as a result of the failure.

Unlike constraints built using dictionaries for getUserMedia, the constraint structures produced by calls to the request() API will always be individual proposed values, rather than min/max ranges. This is because min/max information is already available within the relevant settings, and can be included in calculations before making the call to request(). Therefore, I didn't feel it was necessary to clutter the API surface with that feature.

MediaSettingsRange objects should be used when the setting can generally actually assume a value along a continuum of values. This specification should indicate what the range of values must be for each setting. Given that implementations of various hardware may not exactly map to the same range, an implementation should make a reasonable attempt to translate and scale the hardware's setting onto the mapping provided by this specification. If this is not possible due to a hardware setting supporting (for example) fewer levels of granularity, then the implementation should make the device settings min value reflect the min value reported in this specification, and the same for the max value. Then for values in between the min and max, the implementation may round to the nearest supported value and report that value in the setting.

For example, if the setting is fluxCapacitance, and has a specified range from -10 (min) to 10 (max) in this specification, but the implementation's fluxCapacitance hardware setting only supports values of "off" "medium" and "full", then -10 should be mapped to "off", 10 should map to "full", and 0 should map to "medium". A request to change the value to 3 should be rounded down to the closest supported setting (0).

MediaSettingsList objects should order their enumerated values from minimum to maximum where it makes sense, or in the order defined by the enumerated type where applicable.

MediaSettingsRange interface

readonly attribute any max
The maximum value of this setting.

The type of this value is specific to the setting. Each setting will describe a specific type. That type must be returned for this attribute.

readonly attribute any min
The minimum value of this setting.

The type of this value is specific to the setting. Each setting will describe a specific type. That type must be returned for this attribute.

readonly attribute any initial
The initial value of this setting. When the object associated with this setting is first made available to the application, the current value of the setting should be set to the initial value. For example, in a browsing scenario, if one web site changes this setting and a subsequent web site gets access to this same setting, the setting should have been reset back to its initial value.

The type of this value is specific to the setting. Each setting will describe a specific type. That type must be returned for this attribute.

void request(any value, optional boolean mandatory)
Creates an internal constraint based on the setting name with the provided value, adds that constraint into the pending constraint structure (to the MediaTrackConstraint array by default or replaces an entry in the MediaTrackConstraintSet if the mandatory flag is set) and queues a task (if not already queued) to process the pending constraint structure at the conclusion of this task.

The mandatory parameter defaults to false.

The value parameter type of this method is specific to the setting. Each setting will describe a specific type. That type must be provided for this paramter. If the type does align, then the implementation should throw a TypeError exception.

MediaSettingsList interface

readonly attribute unsigned long length
The length of the enumerated values that this setting may assume.
getter any item(unsigned long index)
Retrieves the value of the indexed enumerated item of this setting. Items should be sorted from min (at index 0) to max where applicable, or in the order listed in the enumerated type otherwise.

The type of this value is specific to the setting. Each setting will describe a specific type. That type must be returned for this attribute.

readonly attribute any initial
The initial value of this setting. When the object associated with this setting is first made available to the application, the current value of the setting should be set to the initial value. For example, in a browsing scenario, if one web site changes this setting and a subsequent web site gets access to this same setting, the setting should have been reset back to its initial value.

The type of this value is specific to the setting. Each setting will describe a specific type. That type must be returned for this attribute.

void request(any value, optional boolean mandatory)
Creates an internal constraint based on the setting name with the provided value, adds that constraint into the pending constraint structure (to the MediaTrackConstraint array by default or replaces an entry in the MediaTrackConstraintSet if the mandatory flag is set) and queues a task (if not already queued) to process the pending constraint structure at the conclusion of this task.

The mandatory parameter defaults to false.

The value parameter type of this method is specific to the setting. Each setting will describe a specific type. That type must be provided for this paramter. If the type does align, then the implementation should throw a TypeError exception.

Basic settings for pictures and video devices

Settings (read/writable) are defined as separate properties from their read-only counterparts. This allows for a variety of benefits:

These are pluralized for compactness and easy identification as a "setting". The more verbose "widthSettings", "horizontalAspectRatioSettings", "orientationSettings", etc., were considered (and may still be considered).

The following settings have been proposed, but are not included in this version to keep the initial set of settings scoped to those that:
  1. cannot be easily computed in post-processing
  2. are not redundant with other settings
  3. are settings found in nearly all devices (common)
  4. can be easily tested for conformance
Each setting also includes a brief explanatory rationale for why it's not included:
  1. width - I've used "dimension" for the setting instead, since resolutions of the camera are nearly always in step-wise pairs of width/height combinations. These are thus an enumerated type rather than a range continuum of possible width/height (independent) pairs.
  2. height - see width explanation
  3. horizontalAspectRatio - easily calculated based on width/height in the dimension values
  4. verticalAspectRatio - see horizontalAspectRatio explanation
  5. orientation - can be easily calculated based on the width/height values and the current rotation
  6. aperatureSize - while more common on digital cameras, not particularly common on webcams (major use-case for this feature)
  7. shutterSpeed - see aperatureSize explanation
  8. denoise - may require specification of the algorithm processing or related image processing filter required to implement.
  9. effects - sounds like a v2 or independent feature (depending on the effect).
  10. faceDetection - sounds like a v2 feature. Can also be done using post-processing techniques (though perhaps not as fast...)
  11. antiShake - sounds like a v2 feature.
  12. geoTagging - this can be independently associated with a recorded picture/video/audio clip using the Geolocation API. Automatically hooking up Geolocation to Media Capture sounds like an exercise for v2 given the possible complications.
  13. highDynamicRange - not sure how this can be specified, or if this is just a v2 feature.
  14. skintoneEnhancement - not a particularly common setting.
  15. shutterSound - Can be accomplished by syncing custom audio playback via the <audio> tag if desired. By default, there will be no sound issued.
  16. redEyeReduction - photo-specific setting. (Could be considered if photo-specific settings are introduced.)
  17. meteringMode - photo-specific setting. (Could be considered if photo-specific settings are introduced.)
  18. iso - photo-specific setting. while more common on digital cameras, not particularly common on webcams (major use-case for this feature)
  19. sceneMode - while more common on digital cameras, not particularly common on webcams (major use-case for this feature)
  20. antiFlicker - not a particularly common setting.
  21. zeroShutterLag - this seems more like a hope than a setting. I'd rather just have implementations make the shutter snap as quickly as possible after takePicture, rather than requiring an opt-in/opt-out for this setting.
The following settings are up for debate in my opinion:
  1. exposure
  2. exposureCompensation (is this the same as exposure?)
  3. autoExposureMode
  4. brightness
  5. contrast
  6. saturation
  7. sharpness
  8. evShift
  9. whiteBalance

Some of the above settings are available as constraints, and so are included in the proposed set of constraints in the last section.

PictureAndVideoSettings mix-in interface

VideoDeviceTrack implements PictureAndVideoSettings;
PictureDeviceTrack implements PictureAndVideoSettings;
readonly attribute MediaSettingsList dimensions
The MediaSettingsList reports values of type VideoDimensionDict. The width/height reported are of the camera's sensor, not reflecting a particular orientation. VideoDimensionDict items in the MediaSettingsList list must be primarily sorted by width (smallest to largest), and then sub-sorted by height (smallest to largest).
readonly attribute unsigned long rotation
The current rotation value in use by the camera. If not supported, the property returns the value 0.
readonly attribute MediaSettingsList? rotations
The MediaSettingsList reports values of type unsigned long. (0-90-180-270) degrees).

Rotation makes me think I could set this to 45 degrees or some such. Maybe there's a better setting name for this. I only want to support right-angles.

readonly attribute float zoom
The current zoom scale value in use by the camera. If the zooms property is not available (not supported), then this property will always return 1.0.

In the case that a camera device supports both optical and digital zoom, does it make sense to have just one property? I expect this to be the "digitalZoom" version, which is more common on devices.

readonly attribute MediaSettingsRange? zooms
The MediaSettingsRange reports values of type float. (initial value is 1. The float value is a scale factor, for example 0.5 is zoomed out by double, while 2.0 is zoomed in by double. Requests should be rounded to the nearest supporting zoom factor by the implementation (when zoom is supported).
readonly attribute VideoFocusModeEnum focusMode
The camera's current focusMode state.
readonly attribute MediaSettingsList? focusModes
The MediaSettingsList reports values of type VideoFocusModeEnum (less the "notavailable" value). The initial value is "auto".
readonly attribute VideoFillLightModeEnum fillLightMode
The camera's current fill light mode.

fillLight seemed more appropriate a term to use for both cameras and photo settings.

readonly attribute MediaSettingsList? fillLightModes
The MediaSettingsList reports values of type VideoFillLightModeEnum (less the "notavailable" value). The initial value is "auto".

VideoDimensionDict dictionary

unsigned long width
A supported camera width (long axis).
unsigned long height
A supported camera height (short axis).

The following enums had many more values in the prior proposal, but in the interest of testing, I've scoped the initial list to those that seem most easily testable.

VideoFocusModeEnum enumeration

notavailable
This camera does not have an option to change focus modes.
auto
The camera auto-focuses.
manual
The camera must be manually focused.

VideoFillLightModeEnum enumeration

notavailable
This camera does not have an option to change fill light modes (e.g., the camera does not have a flash).
auto
The camera's fill light will be enabled when required (typically low light conditions). Otherwise it will be off.
off
The camera's fill light will not be used.
on
The camera's fill light will be turned on until this setting is changed again, or the underlying track object has ended.

Expanded settings for video devices

VideoDeviceTrack partial interface

readonly attribute float framesPerSecond
The camera's currently configured (estimated) framesPerSecond.
readonly attribute MediaSettingsRange? framesPerSeconds
The MediaSettingRange reports values of type float.

I wonder if this should just be a MediaSettingsList with the common values of 15, 30, and 60. Are there really any other values coming from hardware?

Settings for audio devices

My previous proposal included a "bassTone" and "trebleTone" setting value, but on reflection, those settings are more relevant to playback than to microphone device settings. Those settings have been removed.

AudioDeviceTrack partial interface

readonly attribute MediaSettingsRange? levels
The MediaSettingRange reports values of type unsigned long.

Tracking the result of constraint application

MediaConstraintResultEventHandlers mix-in interface

AudioDeviceTrack implements MediaConstraintResultEventHandlers;
VideoDeviceTrack implements MediaConstraintResultEventHandlers;
PictureDeviceTrack implements MediaConstraintResultEventHandlers;
MediaDeviceList implements MediaConstraintResultEventHandlers;
attribute EventHandler onconstrainterror
Register/unregister for "constrainterror" events. The handler should expect to get a ConstraintErrorEvent object as its first parameter. The event is fired asynchronously after [potentially many] settings change requests are made but resulted in one or more failures to meet those constraints. The ConstraintErrorEvent reports the name of the settings that could not be applied.
attribute EventHandler onconstraintsuccess
Register/unregister for "constraintsuccess" events. The handler should expect to get a DeviceEvent object as its first parameter. The event is fired asynchronously after the [potentially many] settings change requests are made and applied successfully. Note, if any one setting change fails, then the "constrainterror" event fires instead. The DeviceEvent will fire on the track making the settings request (with the device attribute referring to the same object), with the exception of the MediaDeviceList (see the MediaDeviceList's select() API).

ConstraintErrorEvent interface

readonly attribute DOMString[] optionalConstraints
A list of optional constraints that failed or succeeded (depending on the event type).
readonly attribute DOMString[] mandatoryConstraints

ConstraintErrorEventInit dictionary

sequence<'DOMString> optionalConstraints
List of optional constraints to populate into the ConstraintErrorEvent object's optionalConstraints readonly attribute.
sequence<'DOMString> mandatoryConstraints
List of mandatory constraints to populate into the ConstraintErrorEvent object's mandatoryConstraints readonly attribute.

Device Lists

One common problem with all my previous proposals, and with the existing model for using getUserMedia to request access to additional devices, is the problem of discovery of multiple devices. As I understand it, the existing recommendation relies on "guessing" by making a second (or third, etc.) request to getUserMedia for access to additional devices. This model has two primary advantages:

First, it ensures privacy by making sure that each device request could be approved by the user. I say "could" because there is no current requirement that the user agent be involved, especially when re-requesting a device type that was already approved, for example, a second "video" device. I surmise that a request for a different class of device ("audio", when exclusive "video" was previously approved), would be cause for an implementation to ask the user for approval.

Second, it ensure privacy by not leaking any information about additional devices until the code has successfully requested a device.

Unfortunately, this model does not provide a means for discovery of additional devices. Such a discovery mechanism could be trivially added to this proposal in the form of a device-specific "totalDevices" property, but there's an opportunity for considering a solution that both streamlines the usability of multiple devices while maintaining the privacy benefits of the current model.

The device list is such a proposal. The device list offers the following benefits:

A device list is merely a list of all AudioDeviceTrack or VideoDeviceTrack objects that are available to the application. Device lists are device-type specific, so there is one device list for all AudioDeviceTrack objects and one device list for all VideoDeviceTrack objects. There is only one instance of each of these lists at any time, and the lists are LIVE (meaning the user agent keeps them up-to-date at all times). Device track objects are added to the list as soon as they are available to the application (e.g., as soon as they are plugged-in) A device track object in the device list will have a readyState set to either LIVE or MUTED). Device tracks are removed from the list when they are unplugged, or otherwise disassociated with their device source such that their readyState changes to ENDED.

Every non-ended device track object will belong to a device list. Of course, the same device track object may also belong to zero or more MediaStreamTrackList objects. The device list provides the one-stop list for all devices of that type regardless of which MediaStream's (if any) the device track objects also belong to.

MediaDeviceList interface

readonly attribute unsigned long length
The number of devices in this list (including those that are MUTED and LIVE.
getter any item(unsigned long index)
Retrieves a device object (an AudioDeviceTrack if this is the audio devices list or a VideoDeviceTrack if this is the video devices list).
readonly attribute unsigned long totalEnabled
The number of devices in this list that whose readyState is in the LIVE state.
void select(MediaTrackConstraints constraints)
Test a set of optional and/or mandatory constraints across the set of devices in the device list, with the goal of selecting a single device. This will queue a task to fire either a "constrainterror" or "constraintsuccess" event depending on the result. The "constraintsuccess" event includes the selected device on the DeviceEvent object's device attribute.

No devices (or their settings) are modified by this API. This API only tests the provided constraints against all the device's capabilities and reports a matching device via the "constraintsuccess" event, or no matches via "constrainterror" event.

attribute EventHandler ondeviceadded
Register/unregister for "deviceadded" events. The handler should expect to get a DeviceEvent object as its first parameter. The event is fired whenever a new video device (or audio device, depending on the device list) becomes available for use. This can happen when a new device is plugged in, for example. Previously ended device tracks are not re-used, and if the user agent is able to re-purpose a physical device for use in the application, it fires the "deviceadded" event providing a new device track object (in its default initial state).
attribute EventHandler ondeviceremoved
Register/unregister for "deviceremoved" events. The handler should expect to get a DeviceEvent object as its first parameter. The event is fired whenever an existing video device (or audio device, depending on the device list) moves into the ENDED state. Note that before dispatching this event, the device in question is removed from the device list.

DeviceEvent interface

readonly attribute MediaStreamTrack device
Contains a reference to the relevant device.

The actual object referenced by the device attribute will be a derived device track object such as an AudioDeviceTrack, VideoDeviceTrack or PictureDeviceTrack.

DeviceEventInit dictionary

MediaStreamTrack device
Video, Audio, or Picture device track used to initialize the "device" property on the DeviceEvent.

Device lists are only accessible from an existing device track object. In other words, the device list itself can only be accessed from one of the devices contained within it (this is an inside-to-outside reference). To help orient the traversal of the list, each device track object includes a (dynamically updated) device index property. If a given device track transitions to the ENDED state, then it will not belong to the device list any longer and its device index property becomes invalid (null); however, the device list itself will still be accessible from that object.

DeviceListAccess mix-in interface

AudioDeviceTrack implements DeviceListAccess;
VideoDeviceTrack implements DeviceListAccess;
readonly attribute MediaDeviceList devices
A reference to the device list for the associated video (or audio) device.
readonly attribute unsigned long? deviceIndex
The current index of this device in the device list. This value can be dynamically changed when other devices are added to (or removed from) the device list. If this device is removed from the device list (because it enters the ENDED state), then the deviceIndex property returns null to signal that this device is not in the device list any longer.

Constraints for navigator.getUserMedia/MediaDeviceList.select

This proposal defines several constraints for use with video and audio devices.

These constraints are applied against the device's range or set of enumerated possible settings, but do not result in a setting change on the device. To change actual settings, use the request() API on each setting.

Video Constraints

The following constraints are applicable to video devices

VideoConstraints dictionary

(unsigned long or MinMaxULongSubConstraint) width
A device that supports the desired width or width range.
(unsigned long or MinMaxULongSubConstraint) height
A device that supports the desired height or height range.
(float or MinMaxFloatSubConstraint) horizontalAspectRatio
A device that supports the desired horizontal aspect ratio (width/height)
(float or MinMaxFloatSubConstraint) verticalAspectRatio
A device that supports the desired vertical aspect ratio (height/width)
unsigned long rotation
A device that supports the desired rotation.
(float or MinMaxFloatSubConstraint) zoom
A device that supports the desired zoom setting.
VideoFocusModeEnum focusMode
A device that supports the desired focus mode.
VideoFillLightModeEnum fillLightMode
A device that supports the desired fill light (flash) mode.
(float or MinMaxFloatSubConstraint) framesPerSecond
A device that supports the desired frames per second.

Audio Constraints

The following constraints are applicable to audio devices

AudioConstraints dictionary

(unsigned long or MinMaxULongSubConstraint) level
A device that supports the desired level or level range.

Common sub-constraint structures

MinMaxULongSubConstraint dictionary

unsigned long max
unsigned long min

MinMaxFloatSubConstraint dictionary

float max
float min

VideoOrientationEnum enumeration

landscape
The long axis of the "effective" video (width/height + rotation) is landscape.
portrait
The long axis of the "effective" video (width/height + rotation) is portrait.

Example usage scenarios

As provided in the 3rd version of this proposal, the following JavaScript examples demonstrate how the Settings APIs defined in this proposal could be used.

Getting access to a video and/or audio device (if available) -- scenario unchanged

navigator.getUserMedia({audio: true, video: true}, gotMedia, failedToGetMedia);

function gotMedia(mediastream) {
   // The recieved mediastream is using its initial settings (it's clean)
}

Previewing the local video/audio in HTML5 video tag -- scenario is unchanged

function gotMedia(mediastream) {
   // objectURL technique
   document.querySelector("video").src = URL.createObjectURL(mediastream, { autoRevoke: true }); // autoRevoke is the default
   // direct-assign technique
   document.querySelector("video").srcObject = mediastream; // Proposed API at this time
}

Applying resolution constraints

function gotMedia(mediastream) {
   var videoDevice = mediastream.videoTracks[0];
   var maxDimensions = videoDevice.dimensions[videoDevice.dimensions.length - 1];
   // Check for 1080p+ support
   if ((maxDimensions.width >= 1920) && (maxDimensions.height >= 1080)) {
      // See if I need to change the current settings...
      if ((videoDevice.width < 1920) && (videoDevice.height < 1080)) {
         videoDevice.dimensions.request(maxDimensions, true);
         videoDevice.onconstrainterror = failureToComply;
      }
   }
   else
      failureToComply();
}

function failureToComply(e) {
   if (e)
      console.error("Device failed to change " + e.mandatoryConstraints[0]); // 'dimension'
   else
      console.error("Device doesn't support at least 1080p");
}

Changing zoom in response to user input:

function gotMedia(mediastream) {
   setupRange( mediastream.videoTracks[0] );
}

function setupRange(videoDevice) {
   // Check to see if the device supports zooming...
   if (videoDevice.zooms) {
      // Set HTML5 range control to min/max values of zoom
      var zoomControl = document.querySelector("input[type=range]");
      zoomControl.min = videoDevice.zooms.min;
      zoomControl.max = videoDevice.zooms.max;
      zoomControl.value = videoDevice.zoom;
      zoomControl.zoomController = videoDevice.zooms; // Store the setting
      zoomControl.onchange = applySettingChanges;
   }
}

function applySettingChanges(e) {
   e.target.zoomController.request(parseFloat(e.target.value), true);
}

Adding the local media tracks into a new media stream -- scenario is unchanged

function gotMedia(mediastream) {
   return new MediaStream( [ mediastream.videoTracks[0], mediastream.audioTracks[0] ]);
}

Take a picture, show the picture in an image tag

function gotMedia(mediastream) {
   var videoDevice = mediastream.videoTracks[0];
   // Check if this device supports a picture mode...
   var pictureDevice = videoDevice.pictureTrack;
   if (pictureDevice) {
       pictureDevice.onpicture = showPicture;
       // Turn on flash only for the snapshot...if available
       if (pictureDevice.fillLightModes) { // If there's an object here, then the flash is supported
          pictureDevice.fillLightModes.request("on", true);
       }
       else
          console.info("Flash not available");
       pictureDevice.takePicture();
   }
}

function showPicture(e) {
   var img = document.querySelector("img");
   img.src = URL.createObjectURL(e.data);
}

Show a newly available device

A newly available device occurs whenever an existing device that was being used by another application (with exclusive access) is relinquished and becomes available for this application to use. Of course, plugging-in a new device also causes a device to become available.

function gotMedia(mediastream) {
   mediastream.videoTracks[0].devices.addEventListener("deviceadded", enableAndShowNewDevice);
}

function enableAndShowNewDevice(e) {
   // Show the new video device as soon as it's available
   // New device is muted when it first becomes available
   e.device.enabled = true;
   var mStream = new MediaStream(e.device);
   document.querySelector("video").srcObject = mStream; // Using the proposed direct-assignment API
}

Show all available video devices

function gotMedia(mediastream) {
   var deviceList = mediastream.videoTracks[0].devices;
   for (var i = 0; i < deviceList.length; i++) {
      var videoDevice = deviceList[i];
      videoDevice.enabled = true;
      // Create a video element and add it to the UI
      var videoTag = document.createElement('video');
      videoTag.srcObject = new MediaStream([videoDevice]);
      document.body.appendChild(videoTag);
   }
}