The Pick Media Intent defines a Web Intent [[!WEBINTENTS]] that enables access to a user's media gallery from inside a Web application. It defines both an Intent action/type pair that selects this operation, and the format of the media data that is returned by services implementing this specification.

This document recasted the previous pure JavaScript APIs version as an API built using Web Intents, while refining the metadata format based on related media data definitions and practical web-based media services.

Introduction

On account of a number of media applications and services accessed from various devices, users tend to maintain their media data in multiple sources including local stroage of devices and web-based media gallleries. For instance, users upload subset of their photos to several different social media galleries with different metadata. For another example, users collect their favorite audio and video resources in several different media galleries which provide inconsistent metadata attributes one another.

A terminology, media object, is used hereunder to denote media data that contain a media content and its related metadata. (See Media dictionary)

This specification enables a Web application to have access to a selected subset of user's media objects, obtained from arbitrary services not known to the Web application. The interactions, brokered using Web Intents [[WEBINTENTS]] are designed in order to maximise the user's security and privacy. Media objects may be sourced from a plurality of sources — both online and local to the user's device — so long as those sources are registered as Intent services with the user agent. It defines a common media object format which services use to provide data to Web applications in a consistent and interoperable manner.

For a Pick Media Intent request, user permission can be naturally asked by a necessary user authentication process that a service implementer MAY provide. Also, users can determine the result media objects to bring to the client with the searching and selecting functionalities that a service implemeter MAY provide.

A set of Security and Privacy Considerations are presented for the discretion of both implementers of Pick Media Intent services and recipients of media data (i.e. Web applications).

The following code illustrates how to obtain media object from a user's media gallery:

        var intent = new Intent({ action:   "http://webintents.org/pick",
                                  type:     "http://w3.org/type/media",
                                  extras:   { search: "olympic",
                                              filters: ["title", "description", "author", "tags"],
                                              limit: 100 }});
        navigator.startActivity(intent, mediaOK, mediaFail);

        function mediaOK (mediaObjectArray) {
            // iterate over the array of media objects to do something useful with them
        }
        function mediaFail (err) {
            // display an error to the user
        }
      

When the above code is run, the user would typically be prompted by her user agent to select a service able to pick media objects (there may be several such services, if she has multiple media galleries). Upon selecting a service, she will be presented with an interface enabling her to choose what media objects are returned to the Web application. Upon completing her choice, an array of the media objects would be returned to the Web application in the mediaOK callback.

There is only one single conformance requirement placed upon the user agent product: a user agent MUST support Web Intents [[!WEBINTENTS]].

The conformance criteria in this specification apply to a single product: the Media service which exposes a Web Intent service that handles Pick Media Intents as defined in this specification.

The data returned by the Media service is described in this specification using [[!WEBIDL]]. When this data is provided using JavaScript, then the Media service MUST do so in a manner consistent with the ECMAScript Bindings defined in the Web IDL specification.

Security and Privacy Considerations

The Intent defined in this specification can be used to find media objects from user's media galleries. The media objects, in the form of photos, videos, voice recordings with related metadata, may contain user's private information. The distribution of this information could potentially compromise the user's privacy. A conforming implementation of this specification should provide a mechanism that protects the user's privacy and this mechanism should ensure that no such information is retrievable without the user's express permission.

Privacy considerations for implementers of the Pick Media Intent

A media service should not provide media objects to Web sites without the express permission of the user. Obtaining the user's express permission to access a set of media objects does not imply that the user has granted permission for the same Web site to access more media objects. A media service should take great care to ensure that the user can clearly see which information is about to be shared, and must not share more information than has been requested by the Web application.

A user agent may have prearranged trust relationships with a specific media service that do not require such user interaction.

Privacy considerations for recipients of media objects

Web sites operators that retrieve media objects using this Intent are denoted as recipients below.

Recipients should only request media objects when necessary, and only use the media objects for the task for which it was provided to them.

Recipients should dispose of media objects once that task is completed, unless expressly permitted to retain it by the user. Recipients should also take measures to protect this information against unauthorised access. If media objects are stored, users should be allowed to update and delete this information.

The recipient of media objects should not retransmit the media objects without the user's express permission. Care should be taken when retransmitting and use of encryption is encouraged.

Recipients should clearly and conspicuously disclose the fact that they are collecting media objects, the purpose of the collection, how long the data is retained, how the data is secured, how the data is shared if it is shared, how users can access, update and delete the data, and any other choices that users have with respect to the data. This disclosure should include an explanation of any exceptions to the guidelines listed above.

Note that even if a user gives permission to share their media objects this can have serious privacy implications for those parties whose media objects are shared, as they may not wish such sharing to occur. This should be considered by Web applications when requesting and using such information.

Additional implementation considerations

Further to the requirements listed in the previous section, implementers of a user agents are also advised to consider the following aspects that can negatively affect the privacy of their users: in certain cases, users can inadvertently grant permission to disclose their media objects to Web sites. In other cases, the content hosted at a certain URL changes in such a way that the previously granted permissions no longer apply as far as the user is concerned. Or the users might simply change their minds.

Predicting or preventing these situations is inherently difficult. Mitigation and in-depth defensive measures are a user agent's responsibility and not prescribed by this specification. However, in designing these measures, implementers are advised to enable user awareness of information sharing, and to provide easy access to user interfaces that enable revocation of permissions that Web applications have to access this Intent.

Intent Description

The action for this Intent is http://webintents.org/pick.

The type for this Intent is http://w3.org/type/media.

When a media service is matched for delivery using these action and type, it MUST respond in one of two ways:

Intent Extras

The Pick Media Intent can be instantiated with an extras field that adheres to the following dictionary.

The MediaIntentExtras dictionary

The MediaIntentExtras dictionary describes the options that the invoker can inform to the service as hints in searching the media objects. None of the fields are mandatory.

DOMString? search
A string which provides a hint to the media service to search media objects. The exact manner in which this hint is exploited is entirely up to the media service.
DOMString[]? filters
An array of string which contain hints to the media service to the DOMString and DOMString[] fields in the Media dictionary that the Web application prefers to use as search criteria. The media service MAY use the values of the filters to search media objects in the media gallery. If a filter name is provided that the media service does not recognise as a field of the Media dictionary, then it MUST ignore it.
unsigned long? limit
By default a media service MAY return as many media objects as the user selects. If limit is specified, the media service MUST NOT return more than limit media objects. The media service SHOULD enforce this limitation in the user interface that it exposes.

Data Format

Upon successful invocation, the media service MUST return an array of Media dictionaries.

The Media dictionary

The Media dictionary captures the properties of a media object. The properties defined in this dictionary present the content (URI by default and Blob as option) of the media data and the related metadata. The properties of the dictionary are defined based on the previous work of metadata survey in Media Ontology WG, the core set, [[MEDIA-ANNOT-REQS]] as well as the common attributes of open APIs provided by typical web-based media services, thereby allowing the data format to be supported across implementations supporting these various media metadata representations.

Additional attributes MAY be included according to the provisions detailed in Extended Media Properties and Parameters.

MediaContent content
A dictionary containing a URI to the content of the media resource. It optionally contains a Blob of the content.
DOMString description
The text description of the content of the media resource.
DOMString? id

A URI identifying a media resource.

The property, identifier, defined in the core set of [Ontology for Media Resources 1.0] can be considered as a representation format.

DOMString title
The title or name given to the media resource.
DOMString type
The MIME type of the media resource (e.g., image/png, video/*) [[RFC2046]].
DOMString? author
The author of the media resource in string.
DOMString? category
The category (genre) of the content of the media resource.
DOMString? copyright
The copyright statement associated with the media resource.
DOMString? fileName
This field's value is not a url, but is the filename (probably not fully qualified, but simply the name of the leaf file) of the resource being passed as assigned by the user. (See WebIntents/MIME Types)
unsigned long? fileSize
The size of the content of the media resource.
Position? location
The location information related to the content of the media resource. [[GEOLOCATION-API]]
DOMString[]? tags

The array of strings representing series of tag string.

MediaContent? thumbnail
A dictionary containing a URI to the thumbnail of the content of the media resource. It optionally contains a Blob of the thumbnail of the content.
Date? createdDate

The creation date of the media resource in Date type.

Date? updatedDate

The last updated date of the media resource in Date type.

DOMString? collection

The name of the collection or album that the media resource belongs to.

unsigned long? duration

The actual duration of the resource. The units are defined to be seconds.

unsigned long? frameRate

The video frame rate. The units are defined to be frames per second.

DOMString? language

The language used in the media resource.

unsigned long? likeCount

The count of the vote in favor of the media resource.

float? rating

The rating value (e.g., customer rating, review, audience appreciation) of the media resource.

unsigned long? ratingCount

The number of ratings made on the media resource.

DOMString? resolution

The frame size of the resource (e.g., width and height of 720 and 480 in px is represented as 720*480+px).

If the usage of this property is frequent, a seperate dictionary can be defined refering to frameSize property defined in [Ontology for Media Resources 1.0]

unsigned long? samplingRate

The audio sampling rate. The units are defined to be samples per second.

Date? publishedDate

The publishing date of the media resource in Date type.

DOMString? publisher

The publisher of the media resource.

unsigned long? viewCount

The access count made on the media resource.

The MediaContent dictionary

The MediaContent dictionary describes a media content reference in detail.

DOMString uri
The URI to the content of the media resource. This attribute is a mandatory property.
Blob? blob
The optional property to contain the media content in Blob format.

The MeidaError dictionary

If the media service encounters an error then it MUST return an error (through postFailure()) using the MediaError dictionary.

DOMString message
A message describing the error.

Extended Media Properties and Parameters

A media service MAY extend the dictionaries described in in the Data Formats section with additional fields. If providing an extended field, a media service MUST prefix its name with X (U+0058 LATIN CAPITAL LETTER X) or use a vendor-specific prefix.

Media Service Example

This section describes an example usage of the Pick Media Intent in media service. It shows the use of the values delivered through MediaIntentExtras dictionary, and the way to return the result array of media objects through Media dictionary.

Accessing MediaIntentExtras Values - Example #1

Media service implementers can retrieve the values delivered in extra field of the intent request. The extra field contains the MediaIntentExtras dictionary:

  <script type="text/javascript">
  if (window.intent) {
      var keyword = window.intent.extra.search;  // a string based hint to search
      var filters = window.intent.extra.filters; // DOMString[] based hint to fields to match
      var limit = window.intent.extra.limit;     // maximum number of objects to return

      // do search relevant media objects based on the extras
      var mediaObjectArray = searchMediaobjects(keyword, filters, limit);
      
      if (mediaObjectArray) {
          window.intent.postResult(mediaObjectArray);
      } else {
          window.intent.postFailure(errorMessage);
      }
  }  
  </script>
          

The use of the properties in MediaIntentExtras dictionary is optional. Service implementers can decide whether these properties are to be used as hints for search.

Creating Media Object - Example #2

Media service implementers can create media objects in the format of the Media dictionary and make an array of media objects to send to client. (using postResult()):

  <script type="text/javascript">
  if (window.intent) {
      var content = {};
      content.uri = "http://example.com/gallery/image00001.png";
       
      var mediaObject = new Object();
      mediaObject.content = content;
      mediaObject.description = "The city view of Seoul";
      mediaObject.id = ""; 
      mediaObject.title = "City of Seoul";
      mediaObject.type = "image/png";  // MIME type of the media content
      mediaObject.author = "J.";

      var mediaObjectArray = new Array();
      
      mediaObjectArray[0] = mediaObject;
      
      if (mediaObjectArray) {
          window.intent.postResult(mediaObjectArray);
      } else {
          window.intent.postFailure(errorMessage);
      }
  }  
  </script>