Proposal: Media Capture and Streams Settings API v6

3.1.1 MediaStreamTrack interface

interface MediaStreamTrack : EventTarget {
             attribute DOMString           id;
    readonly attribute DOMString           kind;
    readonly attribute DOMString           label;
             attribute boolean             enabled;
    readonly attribute TrackReadyStateEnum readyState;
             attribute EventHandler        onstarted;
             attribute EventHandler        onmute;
             attribute EventHandler        onunmute;
             attribute EventHandler        onended;
};

3.1.2 TrackReadyStateEnum enumeration

enum TrackReadyStateEnum {
    "new",
    "live",
    "muted",
    "ended"
};

3.2.1 Track Source API Extensions to MediaStreamTrack

partial interface MediaStreamTrack {
    readonly attribute SourceTypeEnum sourceType;
    readonly attribute DOMString      sourceId;
    void stop ();
};

3.2.2 Track Source Types

The sourceType attribute may have the following states:

enum SourceTypeEnum {
    "none",
    "camera",
    "microphone",
    "photo-camera",
    "readonly",
    "remote"
};

3.3.1 VideoStreamTrack interface

Video tracks may be instantiated with optional media track constraints. These constraints can be later modified on the track as needed by the application, or created after-the-fact if the initial constraints are unknown to the application.

[Constructor(optional MediaTrackConstraints videoConstraints)]
interface VideoStreamTrack : MediaStreamTrack {
    static sequence<DOMString> getSourceIds ();
    void                       takePhoto ();
             attribute EventHandler onphoto;
             attribute EventHandler onphotoerror;
};

3.3.2 Photo-related Event Definitions

BlobEvent interface

[Constructor(DOMString type, optional BlobEventInit blobInitDict)]
interface BlobEvent : Event {
    readonly attribute Blob data;
};

BlobEventInit dictionary

dictionary BlobEventInit : EventInit {
    Blob data;
};

3.3.3 AudioStreamTrack interface

[Constructor]
interface AudioStreamTrack : MediaStreamTrack {
    static sequence<DOMString> getSourceIds ();
};

4.1.1 Source State API Extensions to VideoStreamTrack

partial interface VideoStreamTrack {
    readonly attribute unsigned long?             width;
    readonly attribute unsigned long?             height;
    readonly attribute float?                     frameRate;
    readonly attribute VideoFacingModeEnum?       facingMode;
    readonly attribute float?                     zoom;
    readonly attribute VideoFocusModeEnum?        focusMode;
    readonly attribute VideoFillLightModeEnum?    fillLightMode;
    readonly attribute VideoWhiteBalanceModeEnum? whiteBalanceMode;
    readonly attribute unsigned long?             brightness;
    readonly attribute unsigned long?             contrast;
    readonly attribute unsigned long?             saturation;
    readonly attribute unsigned long?             sharpness;
    readonly attribute unsigned long?             photoWidth;
    readonly attribute unsigned long?             photoHeight;
    readonly attribute PhotoExposureModeEnum?     exposureMode;
    readonly attribute PhotoISOModeEnum?          isoMode;
};

4.1.2 Video Source State Supporting Enumerations

VideoFacingModeEnum enumeration

enum VideoFacingModeEnum {
    "notavailable",
    "user",
    "environment"
};

VideoFocusModeEnum enumeration

enum VideoFocusModeEnum {
    "notavailable",
    "auto",
    "manual"
};

VideoFillLightModeEnum enumeration

enum VideoFillLightModeEnum {
    "notavailable",
    "auto",
    "off",
    "flash",
    "on"
};

VideoWhiteBalanceModeEnum enumeration

enum VideoWhiteBalanceModeEnum {
    "notavailable",
    "auto",
    "incandescent",
    "cool-fluorescent",
    "warm-fluorescent",
    "daylight",
    "cloudy",
    "twilight",
    "shade"
};

PhotoExposureModeEnum enumeration

enum PhotoExposureModeEnum {
    "notavailable",
    "auto",
    "frame-average",
    "center-weighted",
    "spot-metering"
};

PhotoISOModeEnum enumeration

enum PhotoISOModeEnum {
    "notavailable",
    "auto",
    "100",
    "200",
    "400",
    "800",
    "1250"
};

4.2.1 Source State API Extensions to AudioStreamTrack

partial interface AudioStreamTrack {
    readonly attribute unsigned long? volume;
    readonly attribute float?         gain;
};

4.3.1 Event Handlers and Object Definitions

The following event handler is added to the generic MediaStreamTrack interface.

partial interface MediaStreamTrack {
             attribute EventHandler onstatechanged;
};

The following define the MediaStreamTrackStateEvent object and related initializer.

[Constructor(DOMString type, optional MediaStreamTrackStateEventInit eventInitDict)]
interface MediaStreamTrackStateEvent : Event {
    readonly attribute DOMString[] states;
};

The initializer for the above-defined event type:

dictionary MediaStreamTrackStateEventInit : EventInit {
    sequence<DOMString> states;
};

5.1.1 Methods

getCapability

If a capability is requested that does not have a corresponding state on the track-type, then a null value is returned (e.g., a VideoStreamTrack requests the "gain" capability. Since "gain" is not a state supported by video stream tracks, this API will return null).

Given that implementations of various hardware may not exactly map to the same range, an implementation should make a reasonable attempt to translate and scale the hardware's setting onto the mapping provided by this specification. If this is not possible due to the user agent's inability to retrieve a given capapbility from a source, then for CapabilityRange-typed capabilities, the min and max fields will not be present on the returned dictionary, and the supported field will be false. For CapabilityList-typed capabilities, a suitable "notavailable" value will be the sole capability in the list.

Note

An example of the user agent providing an alternative mapping: if a source supports a hypothetical fluxCapacitance state whose type is a CapabilityRange, and the state is defined in this specification to be the range from -10 (min) to 10 (max), but the source's (hardware setting) for fluxCapacitance only supports values of "off" "medium" and "full", then the user agent should map the range value of -10 to "off", 10 should map to "full", and 0 should map to "medium". Constraints imposing a strict value of 3 will cause the user agent to attempt to set the value of "medium" on the hardware, and return a fluxCapacitance state of 0, the closest supported setting. No error event is raised in this scenario.

CapabilityList objects should order their enumerated values from minimum to maximum where it makes sense, or in the order defined by the enumerated type where applicable.

See the AllVideoCapabilities and AllAudioCapabilities dictionary for details on the expected types for the various supported state names.

Parameter	Type	Nullable	Optional	Description
stateName	DOMString	?	?	The name of the source state for which the range of expected values should be returned.

Return type: (CapabilityRange or CapabilityList)

capabilities

Returns a dictionary with all of the capabilities for the track type. If the track type is VideoStreamTrack, the AllVideoCapabilities dictionary is returned. If the track type is AudioStreamTrack, the AllAudioCapabilities dictionary is returned.

The dictionaries are populated as if each state were requested individually using getCapability(), and the results of that API are assigned as the value of each stateName in the dictionary. Notably, the returned values

No parameters.

Return type: (AllVideoCapabilities or AllAudioCapabilities)

5.2.1 Dictionary CapabilityRange Members

max of type any

The maximum value of this capability.

The type of this value is specific to the capability as noted in the table for getCapability.

If the related capability is not supported by the source, then this field will not be provided by the user agent (it will be undefined).

min of type any

The minimum value of this capability.

The type of this value is specific to the capability as noted in the table for getCapability.

If the related capability is not supported by the source, then this field will not be provided by the user agent (it will be undefined).

supported of type boolean

Returns the value true if the capability is supported, false otherwise.

5.2.2 Dictionary AllVideoCapabilities Members

sourceType of type CapabilityList, nullable

The available sourceType options (SourceTypeEnum) on the current source.

sourceId of type CapabilityList, nullable

The available source identifiers of the current source--this will always return a list with a single identifier (that of the current source). Note, to get a list of other available source identifiers, use the static getSourceIds method.

width of type CapabilityRange, nullable

The range should span the video source's pre-set width values with min being the smallest width, and max the largest width. The type of the min/max values are unsigned long.

height of type CapabilityRange, nullable

The range should span the video source's pre-set height values with min being the smallest width, and max the largest width. The type of the min/max values are unsigned long.

frameRate of type CapabilityRange, nullable

The supported range of frame rates on the source. The type of the min/max values are float.

facingMode of type CapabilityList, nullable

The available video facing options (VideoFacingModeEnum) on the source.

zoom of type CapabilityRange, nullable

The supported zoom range on the source. The type of the min/max/initial values are float. The initial value is 1. The float value is a scale factor, for example 0.5 is zoomed out by double, while 2.0 is zoomed in by double. Requests should be rounded to the nearest supporting zoom factor by the implementation (when zoom is supported).

focusMode of type CapabilityList, nullable

The available focus mode options (VideoFocusModeEnum) on the source.

fillLightMode of type CapabilityList, nullable

The available fill light mode options (VideoFillLightModeEnum) on the source.

whiteBalanceMode of type CapabilityList, nullable

The available white-balance mode options (VideoWhiteBalanceModeEnum) on the source.

brightness of type CapabilityRange, nullable

The supported range of brightness on the source. The type of the min/max values are unsigned long.

contrast of type CapabilityRange, nullable

The supported range of contrast on the source. The type of the min/max values are unsigned long.

saturation of type CapabilityRange, nullable

The supported range of saturation on the source. The type of the min/max values are unsigned long.

sharpness of type CapabilityRange, nullable

The supported range of sharpness on the source. The type of the min/max values are unsigned long.

photoWidth of type CapabilityRange, nullable

The range should span the video source's high-resolution photo-mode pre-set width values with min being the smallest width, and max the largest width. The type of the min/max/initial values are unsigned long.

photoHeight of type CapabilityRange, nullable

The range should span the video source's high-resolution photo-mode pre-set height values with min being the smallest width, and max the largest width. The type of the min/max/initial values are unsigned long.

exposureMode of type CapabilityList, nullable

The available exposure mode options (PhotoExposureModeEnum) on the source.

isoMode of type CapabilityList, nullable

The available ISO mode options (PhotoISOModeEnum) on the source.

5.2.3 Dictionary AllAudioCapabilities Members

sourceType of type CapabilityList, nullable

The available sourceType options (SourceTypeEnum) on the current source.

sourceId of type CapabilityList, nullable

The available source identifiers of the current source--this will always return a list with a single identifier (that of the current source). Note, to get a list of other available source identifiers, use the static getSourceIds method.

volume of type CapabilityRange, nullable

The supported range of output volume percentages on the source. The type of the min/max values are unsigned long.

gain of type CapabilityRange, nullable

The supported gain range on the source. The type of the min/max values are float.

6.2.1 Attributes

onoverconstrained of type EventHandler

Register an event handler for the "overconstrained" event. This event fires asynchronously for each affected track (when multiple tracks share the same source) after the user agent has evaluated the current constraints against a given sourceId and is not able to configure the source within the limitations established by the union of imposed constraints.

This event may also fire when takePhoto is called and the source cannot record/encode an image due to over-constrained or conflicting constraints of those uniquely related to sourceTypes of type "photo-camera".

Due to being over-constrained, the user agent must transition the source to the "armed" mode, which may result in also dispatching one or more "muted" events to affected tracks.

The affected track(s) will remain un-usable (in the "muted" readyState) until the application adjusts the constraints to accommodate the source's capabilities.

The "overconstrained" event is a simple event of type Event; it carries no information about which constraints caused the source to be over-constrained (the application has all the necessary APIs to figure it out).

6.2.2 Methods

getConstraint

Retrieves a specific named constraint value from the track. The named constraints are the same names used for the capabilities API, and also are the same names used for the source's state attributes.

Returns one of the following types:

null

If no constraint matching the provided constraintName exists in the respective optional or mandatory set on this track.

sequence<MediaTrackConstraint>

If the mandatory flag is false and there is at least one optional matching constraint name defined on this track.

Each MediaTrackConstraint result in the list will contain a key which matches the requested constraintName parameter, and whose value will either be a primitive value, or a MinMaxConstraint object.

The returned list will be ordered from most important-to-satisfy at index 0, to the least-important-to-satisfy optional constraint.

Note

Example: Given a track with an internal constraint structure: { mandatory: { width: { min: 640 }, height: { min: 480 } }, optional: [ { width: 650 }, { width: { min: 650, max: 800 }}, { frameRate: 60 }, { fillLightMode: "off" }, { facingMode: "user" } ] } and a request for getConstraint("width"), the following list would be returned: [ { width: 650 }, { width: { min: 650, max: 800 }} ]

MinMaxConstraint

If the mandatory flag is true, and the requested constraint is defined in the mandatory MediaTrackConstraintSet associated with this track, and the value of the constraint is a min/max range object.

primitive_value

If the mandatory flag is true, and the requested constraint is defined in the mandatory MediaTrackConstraintSet associated with this track, and the value of the constraint is a primitive value (DOMString, unsigned long, float, etc.).

Parameter	Type	Nullable	Optional	Description
constraintName	DOMString	?	?	The name of the setting for which the current value of that setting should be returned
mandatory	boolean = false	?	?	true to indicate that the constraint should be looked up in the mandatory set of constraints, otherwise, the constraintName should be retrieved from the optional list of constraints.

Return type: any

setConstraint

This method updates the value of a same-named existing constraint (if found) in either the mandatory or optional list, and otherwise sets the new constraint.

This method searches the list of optional constraints from index 0 (highest priority) to the end of the list (lowest priority) looking for matching constraints. Therefore, for multiple same-named optional constraints, this method will only update the value of the highest-priority matching constraint.

If the mandatory flag is false and the constraint is not found in the list of optional constraints, then a new optional constraint is created and appended to the end of the list (thus having lowest priority).

Note

Note: This behavior allows applications to iteratively call setConstraint and have their constraints added in the order specified in the source.

Parameter	Type	Nullable	Optional	Description
constraintName	DOMString	?	?	The name of the constraint to set.
constraintValue	any	?	?	Either a primitive value (float/DOMString/etc), or a MinMaxConstraint dictionary.
mandatory	boolean = false	?	?	A flag indicating whether this constraint should be applied to the optional or mandatory constraints.

Return type: void

constraints

Returns the complete constraints object associated with the track. If no mandatory constraints have been defined, the mandatory field will not be present (it will be undefined). If no optional constraints have been defined, the optional field will not be present (it will be undefined). If neither optional, nor mandatory constraints have been created, the value null is returned.

No parameters.

Return type: MediaTrackConstraints, nullable

applyConstraints

This API will replace all existing constraints with the provided constraints (if existing constraints exist). Otherwise, it will apply the newly provided constraints to the track.

Parameter	Type	Nullable	Optional	Description
constraints	MediaTrackConstraints	?	?	A new constraint structure to apply to this track.

Return type: void

prependConstraint

Prepends (inserts before the start of the list) the provided constraint name and value. This method does not consider whether a same-named constraint already exists in the optional constraints list.

This method applies exclusively to optional constraints; it does not modify mandatory constraints.

This method is a convenience API for programmatically building constraint structures.

Parameter	Type	Nullable	Optional	Description
constraintName	DOMString	?	?	The name of the constraint to prepend to the list of optional constraints.
constraintValue	any	?	?	Either a primitive value (float/DOMString/etc), or a MinMaxConstraint dictionary.

Return type: void

appendConstraint

Appends (at the end of the list) the provided constraint name and value. This method does not consider whether a same-named constraint already exists in the optional constraints list.

This method applies exclusively to optional constraints; it does not modify mandatory constraints.

This method is a convenience API for programmatically building constraint structures.

Parameter	Type	Nullable	Optional	Description
constraintName	DOMString	?	?	The name of the constraint to append to the list of optional constraints.
constraintValue	any	?	?	Either a primitive value (float/DOMString/etc), or a MinMaxConstraint dictionary.

Return type: void

6.2.3 Constraint Definitions and Related Structures

The following constraint names are defined to apply to both VideoStreamTrack and AudioStreamTrack objects:

The following constraint names are defined to apply only to VideoStreamTrack objects:

The following constraint names are defined to apply only to AudioStreamTrack objects:

For constraints that accept ranges, the MinMaxConstraint dictionary is also defined. Note, that the type of the value associated with min and max must be the same for both. The specific types associated with min and max are defined differently for each constraint name.

dictionary MinMaxConstraint {
    any max;
    any min;
};