The Device APIs Working Group is currently not progressing the approach outlined in this draft. Please treat this document with caution and do not reference it or use it as the basis for implementation. The domain covered by this document is still within the scope of the Working Group as defined in its Charter. The Working Group may resume this work or adopt an alternative approach depending on the interest of WG members and implementers.
The Pick Media Intent defines a Web Intent [[!WEBINTENTS]] that enables access to a user's media gallery from inside a Web application. It defines both an Intent action/type pair that selects this operation, and the format of the media data that is returned by services implementing this specification.
This document recasts the previous pure JavaScript APIs version as an API built using Web Intents, while refining the metadata format based on related media data definitions and practical web-based media services.
On account of a number of media applications and services accessed from various devices, users tend to maintain their media data in multiple sources including local storage of devices and web-based media galleries. For instance, users upload subset of their photos to several different social media galleries with different metadata. For another example, users collect their favourite audio and video resources in several different media galleries which provide inconsistent metadata attributes one another.
A terminology, media object, is used hereunder to denote media data that contain a media content and its related metadata. (See Media dictionary)
This specification enables a Web application to have access to a selected subset of user's media objects, obtained from arbitrary services not known to the Web application. The interactions, brokered using Web Intents [[WEBINTENTS]] are designed in order to maximise the user's security and privacy. Media objects may be sourced from a plurality of sources — both online and local to the user's device — so long as those sources are registered as Intent services with the user agent. It defines a common media object format which services use to provide data to Web applications in a consistent and interoperable manner.
For a Pick Media Intent request, user permission can be naturally asked by a necessary user authentication process that a service implementer MAY provide. Also, users can determine the result media objects to bring to the client with the searching and selecting functionalities that a service implementer MAY provide.
A set of Security and Privacy Considerations are presented for the discretion of both implementers of Pick Media Intent services and recipients of media data (i.e. Web applications).
The following code illustrates how to obtain media object from a user's media gallery:
var intent = new Intent({ action: "http://intents.w3.org/pick",
type: "http://intents.w3.org/type/media",
extras: { search: "Olympic",
filters: ["title", "description", "author", "tags"],
limit: 100 }});
navigator.startActivity(intent, mediaOK, mediaFail);
function mediaOK (mediaObjectArray) {
// iterate over the array of media objects to do something useful with them
}
function mediaFail (err) {
// display an error to the user
}
When the above code is run, the user would typically be prompted by her user agent to select
a service able to pick media objects (there may be several such services, if she has multiple media galleries). Upon selecting a service, she will be presented with an interface enabling her
to choose what media objects are returned to the Web application. Upon completing her
choice, an array of the media objects would be returned to the Web application in the mediaOK
callback.
There is only one single conformance requirement placed upon the user agent product: a user agent MUST support Web Intents [[!WEBINTENTS]].
The conformance criteria in this specification apply to a single product: the Media service which exposes a Web Intent service that handles Pick Media Intents as defined in this specification.
The data returned by the Media service is described in this specification using [[!WEBIDL]]. When this data is provided using JavaScript, then the Media service MUST do so in a manner consistent with the ECMAScript Bindings defined in the Web IDL specification.
The Intent defined in this specification can be used to find media objects from user's media galleries. The media objects, in the form of photos, videos, voice recordings with related metadata, may contain user's private information. The distribution of this information could potentially compromise the user's privacy. A conforming implementation of this specification should provide a mechanism that protects the user's privacy and this mechanism should ensure that no such information is retrievable without the user's express permission.
A media service should not provide media objects to Web sites without the express permission of the user. Obtaining the user's express permission to access a set of media objects does not imply that the user has granted permission for the same Web site to access more media objects. A media service should take great care to ensure that the user can clearly see which information is about to be shared, and must not share more information than has been requested by the Web application.
A user agent may have prearranged trust relationships with a specific media service that do not require such user interaction.
Web sites operators that retrieve media objects using this Intent are denoted as recipients below.
Recipients should only request media objects when necessary, and only use the media objects for the task for which it was provided to them.
Recipients should dispose of media objects once that task is completed, unless expressly permitted to retain it by the user. Recipients should also take measures to protect this information against unauthorised access. If media objects are stored, users should be allowed to update and delete this information.
The recipient of media objects should not retransmit the media objects without the user's express permission. Care should be taken when retransmitting and use of encryption is encouraged.
Recipients should clearly and conspicuously disclose the fact that they are collecting media objects, the purpose of the collection, how long the data is retained, how the data is secured, how the data is shared if it is shared, how users can access, update and delete the data, and any other choices that users have with respect to the data. This disclosure should include an explanation of any exceptions to the guidelines listed above.
Note that even if a user gives permission to share their media objects this can have serious privacy implications for those parties whose media objects are shared, as they may not wish such sharing to occur. This should be considered by Web applications when requesting and using such information.
Further to the requirements listed in the previous section, implementers of a user agents are also advised to consider the following aspects that can negatively affect the privacy of their users: in certain cases, users can inadvertently grant permission to disclose their media objects to Web sites. In other cases, the content hosted at a certain URL changes in such a way that the previously granted permissions no longer apply as far as the user is concerned. Or the users might simply change their minds.
Predicting or preventing these situations is inherently difficult. Mitigation and in-depth defensive measures are a user agent's responsibility and not prescribed by this specification. However, in designing these measures, implementers are advised to enable user awareness of information sharing, and to provide easy access to user interfaces that enable revocation of permissions that Web applications have to access this Intent.
The action for this Intent is http://intents.w3.org/pick.
The type for this Intent is http://intents.w3.org/type/media.
When a media service is matched for delivery using these action and type, it MUST respond in one of two ways:
postResult()) matching the data format defined below.
postFailure()) matching the error data format defined below. The media service
MUST NOT treat the user selecting zero media object or cancelling the service as error conditions.
The Pick Media Intent can be instantiated with an extras field that adheres to the
following dictionary.
The MediaIntentExtras dictionary describes the options that the invoker can inform to the service as hints in searching the media objects. None of the fields are mandatory.
limit
is specified, the media service MUST NOT return more than limit media objects. The
media service SHOULD enforce this limitation in the user interface that it exposes.
Upon successful invocation, the media service MUST return an array of Media dictionaries.
The Media dictionary captures the properties of a media object. The properties defined in this dictionary present the content (URI by default and Blob as option) of the media data and the related metadata. The properties of the dictionary are defined based on the previous work of metadata survey in Media Ontology WG, the core set, [[MEDIA-ANNOT-REQS]] as well as the common attributes of open APIs provided by typical web-based media services, thereby allowing the data format to be supported across implementations supporting these various media metadata representations.
Additional attributes MAY be included according to the provisions detailed in Extended Media Properties and Parameters.
The name of the attribute represents various contributors, e.g. creator, artist, actoress, etc., in the given media resouce. Hence, the name of the attribute can be changed based on the discussion.
The usage of the attribute in deployed web services is low (see Metadata properties table.) However, the use case exists in audio applications. The retention of the attribue will be discussed in the group.
The name of the attribute represents various sets of media resources, e.g. music album, photo album, video collection, etc. Hence, the name of the attribute can be changed based on the discussion.
Although, the original purpose of the attribute is to denote the file path of user's local storage, any additional URL information can be delivered in this attribute.
The usage of the attribute is confined to audio type. (see Metadata properties table.) However, the use case exists in audio applications. The retention of the attribue will be discussed in the group.
The usage of the attribute in deployed web services is low (see Metadata properties table.) However, the use case exists in audio and video applications. The retention of the attribue will be discussed in the group.
The MediaContent dictionary describes a media content reference in detail.
The Resolution dictionary describes a display resolution of the given media object. It is usually denoted as "width × height", with the units in pixels. For example, the HD resolution can be specified as "1280 × 720" meaning the width is 1280 pixels and the height is 720 pixels.
If the media service encounters an error then it MUST return an error
(through postFailure()) using the MediaError dictionary.
A media service MAY extend the dictionaries described in in the Data Formats section with
additional fields. If providing an extended field, a media service MUST prefix its name
with X (U+0058 LATIN CAPITAL LETTER X) or use a vendor-specific prefix.
This section describes an example usage of the Pick Media Intent in media service. It shows the use of the values delivered through MediaIntentExtras dictionary, and the way to return the result array of media objects through Media dictionary.
Media service implementers can retrieve the values delivered in extra field of the intent request. The extra field contains the MediaIntentExtras dictionary:
<script type="text/javascript">
if (window.intent) {
var keyword = window.intent.extra.search; // a string based hint to search
var filters = window.intent.extra.filters; // DOMString[] based hint to fields to match
var limit = window.intent.extra.limit; // maximum number of objects to return
// do search relevant media objects based on the extras
var mediaObjectArray = searchMediaObjects(keyword, filters, limit);
if (mediaObjectArray) {
window.intent.postResult(mediaObjectArray);
} else {
window.intent.postFailure(errorMessage);
}
}
</script>
The use of the properties in MediaIntentExtras dictionary is optional. Service implementers can decide whether these properties are to be used as hints for search.
Media service implementers can create media objects in the format of the Media dictionary and make an array of media objects to send to client. (using postResult()):
<script type="text/javascript">
if (window.intent) {
var content = {};
content.uri = "http://example.com/gallery/image00001.png";
var mediaObject = new Object();
mediaObject.content = content;
mediaObject.description = "The city view of Seoul";
mediaObject.title = "City of Seoul";
mediaObject.type = "image/png"; // MIME type of the media content
mediaObject.author = "J.";
var mediaObjectArray = new Array();
mediaObjectArray[0] = mediaObject;
if (mediaObjectArray) {
window.intent.postResult(mediaObjectArray);
} else {
window.intent.postFailure(errorMessage);
}
}
</script>
This section describes the critera of the metadata selection and shows the selected properties and excluded properties under ongoing discussion. The following table shows the usage of the metadata properties (see the Media dictionary) in several platforms and web services.
| Property | Media | Windows | iOS | Ubuntu | Tizen | Android | SoundCloud | Billboard | YouTube | DailyMotion | flickr | google+ | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Selected Property | |||||||||||||
| content | AVI | o | o | o | o | o | o | o | o | o | o | o | o |
| title | AVI | o | o | o | o | o | o | o | o | o | o | o | |
| description | AVI | o | o | o | o | o | o | o | o | ||||
| type | AVI | o | o | o | o | o | |||||||
| author | AVI | o | o | o | o | o | o | o | o | o | o | o | o |
| publisher | AVI | o | o | o | o | o | o | o | o | o | o | ||
| composer | A | o | o | ||||||||||
| collection | AI | o | o | o | o | o | o | o | o | o | o | ||
| genre | AV | o | o | o | o | o | o | o | o | o | |||
| copyright | AVI | o | o | o | o | ||||||||
| path | AVI | o | o | o | o | o | o | ||||||
| size | AVI | o | o | o | o | o | o | ||||||
| location | VI | o | o | o | o | o | |||||||
| tags | AVI | o | o | o | o | o | o | o | o | ||||
| thumbnail | VI | o | o | o | |||||||||
| date | AVI | o | o | o | o | o | o | o | |||||
| publishDate | AVI | o | o | o | o | o | o | o | o | ||||
| duartion | AV | o | o | o | o | o | o | o | o | o | |||
| resolution | VI | o | o | o | o | o | o | o | o | ||||
| trackNumber | A | o | o | o | o | o | o | ||||||
| subtitle | AV | o | o | o | o | o | o | ||||||
| likeCount | AVI | o | o | o | o | o | o | ||||||
| viewCount | AVI | o | o | o | o | o | o | ||||||
| rating | AVI | o | o | o | o | ||||||||
| Excluded Property | |||||||||||||
| id | AVI | o | |||||||||||
| bitRate | AV | o | o | o | o | ||||||||
| audioCodec | AV | o | o | ||||||||||
| videoCodec | V | o | o | o | |||||||||
| frameRate | V | o | |||||||||||
| language | AV | o | o | ||||||||||
| ratingCount | AVI | ||||||||||||
| samplingRate | A | o | o | ||||||||||
| bitPerMinute | A | o | o | o | |||||||||
| dislikeCount | AVI | o | |||||||||||
* A: Audio, V: Video, I: Image
The following properties are excluded from the Media dictionary definition due to the lack of necessity in most of the applications and services. However, the retention of the properties can be considered upon any strong use cases under ongoing discussion in the group.