This document describes a HTTP-based protocol for clients and servers to be able to efficiently retrieve large Linked Data Platform Resource representations by splitting up the responses into separate URL-addressable page resources.

UNDER CONSTRUCTION

Introduction

This specification provides a widely re-usable pattern to make the state of a large paged HTTP resource available as a list of smaller subset resources (pages) whose representations are less burdensome for servers to form. Paged resources must be LDP Resources (LDPRs) [[!LDP]]. Any LDPR can be paged, but certain aspects of paging like ordering are only well-defined for particular sub-classes of LDPRs, like LDP-RSs or LDPCs.

When a client attempts to retrieve a paged resource, the server either redirects the client to a "first page" resource or provides the representation of the "first page" resource in its response, depending on the client's request preferences. Paging-aware clients can save the latency of making a second request to retrieve the first page when the server supports this, in addition to knowing how to combine pages. In either case, the response containing the first page's representation includes a link to another page in the sequence, as do all other non-terminal pages. LDP Paging defines a mechanism by which clients can learn that the paged resource has been changed so that they can, for example, abandon a page traversal as early as possible. A detailed example of paging is provided in .

When a paged resource is also an LDPC, some additional efficiencies become possible in cases where the server predictably assigns members to pages and is able to communicate its assignment algorithm to clients. defines a facility to communicate the sort order of member-to-page assignments, to handle that common implementation algorithm.

For context and background, it could be useful to read Linked Data Platform Use Case and Requirements [[LDP-UCR]].

Terminology

Terms re-used from the Linked Data Platform

This section is non-normative. It summarizes a subset of terms formally defined in [[LDP]] for the reader's convenience.

LDP server
A conforming HTTP server [[RFC7230]] that follows the rules defined by [[LDP]] when it is serving LDPRs and LDPCs.

LDP client
A conforming HTTP client [[RFC7230]] that follows the rules defined by [[LDP]] when interacting with a LDP server.

Linked Data Platform Resource (LDPR)
A HTTP resource whose state is represented in any way that conforms to the patterns and conventions in [[LDP]].

Linked Data Platform RDF Source (LDP-RS)
An LDPR whose state is fully represented in RDF.

Linked Data Platform Container (LDPC)
A LDP-RS representing a collection of LDPRs linked by membership triples and containment triples.

Membership triples
A set of triples that lists an LDPC's members.

Containment triples
A set of triples, maintained by an LDPC, that lists documents created by the LDPC but not yet deleted.

Terms normatively defined by this specification

The following terminology is based on W3C's Architecture of the World Wide Web [[!WEBARCH]], Hyper-text Transfer Protocol ([[!RFC7230]], [[!RFC7231]]) and Linked Data Platform [[!LDP]].

Paged resource
A LDPR whose representation may be too large for a server to construct in a single HTTP response (e.g. without running out of memory), for which an LDP Paging server offers a sequence of single-page LDPRs.

When the representations of the sequence's resources are combined by a client, the client has a (potentially incomplete or incoherent) copy of the paged resource's state. If a paged resource P is a LDP-RS and is broken into a sequence of pages (in-sequence page resources) P1, P2, ...,Pn, the representation of each Pi contains a subset of the triples in P. LDP allows paging of resources other than LDP-RSs, but does not specify how clients combine their representations. See for additional details.

For readers familiar with paged feeds [[RFC5005]], a paged resource is similar to a logical feed. Any resource could be considered to be a paged resource consisting of exactly one page, although there is no known advantage in doing so.

In-sequence page resource
One of a sequence of related LDPRs P1 (first), P2, ...,Pn (last), each of which contains a subset of the state of another resource P, called the paged resource. For readers familiar with paged feeds [[RFC5005]], an in-sequence page resource is similar to a feed document and the same coherency/completeness considerations apply. LDP provides no guarantees that the sequence of page resources is stable.

Note: the choice of terms was designed to help authors and readers clearly differentiate between the resource being paged, and the individual page resources, in cases where both are mentioned in close proximity.

Note: page sequences are described and named with respect to how they are traversed starting from the paged resource, with "first" being "nearest" to the paged resource, "last" the "furthest" from the paged resource (using only "forward" traversal), and so on. This does not imply anything more; the choice is arbitrary. For example, following forward links does not imply that resources later in the sequence are newer; the forward direction might correspond to moving forward or backward in time or along some other dimension, but any such relationship is server-specific It is not implied by LDP Paging, absent additional concrete assertions like those defined later for LDPC representations.

first page link
A link to the first in-sequence page resource P1 (first) of a paged resource P. The first page is the one that a LDP paging server redirects to (303 response) or provides the representation of (2NN response) in response to a retrieval request for the paged resource's URI; it is, in that sense, nearest to the paged resource P. Syntactically, a HTTP Link <P1>; rel='first' header [[!RFC5988]].

next page link
A link to the next in-sequence page resource of a paged resource P. Syntactically, a HTTP Link <Pi>; rel='next' header [[!RFC5988]][[!HTML5]] where the context URI identifies some Pi=1 (first)...n-1 (next to last) and the target URI identifies Pi+1.

last page link
A link to the last in-sequence page resource Pn (last) of a paged resource P. The last page is the page that terminates a forward traversal starting from the first page link (or from any later point in the sequence) because it contains no next page link; it is, in that sense, furthest from the paged resource P. Syntactically, a HTTP Link <Pn>; rel='last' header [[!RFC5988]].

previous page link
A link to the previous in-sequence page resource of a paged resource P. Syntactically, a HTTP Link <Pi>; rel='prev' header [[!RFC5988]][[!HTML5]] where the context URI identifies some Pi=2...n (last) and the target URI identifies Pi-1.

paging link
Any of the links defined by LDP Paging: first page links, next page links, last page links, previous page links.

forward traversal
The process of following next page links from some starting point.

Note: "forward" should not be read to mean anything more than a name for one direction through the sequence. For example, following forward links does not imply that resources later in the sequence are newer; the forward direction might correspond to moving forward in time, but any such relationship is server-specific not implied by LDP Paging absent additional concrete assertions like those defined later for LDPC representations.

backward traversal
The process of following previous page links from some starting point.

Note: "backward" should not be read to mean anything more than a name for one direction through the sequence.

Conventions Used in This Document

The namespace for LDP Paging is http://www.w3.org/ns/ldp#.

Sample resource representations are provided in text/turtle format [[turtle]].

Commonly used namespace prefixes:

	
	@prefix dcterms: <http://purl.org/dc/terms/>.
	@prefix foaf:    <http://xmlns.com/foaf/0.1/>.
	@prefix rdf:     <http://www.w3.org/1999/02/22-rdf-syntax-ns#>.
	@prefix ldp:     <http://www.w3.org/ns/ldp#>.
	

The status of the sections of Linked Data Platform Paging 1.0 (this document) is as follows:

A conforming LDP Paging client is a conforming LDP client [[!LDP]] that follows the rules defined by LDP Paging.

A conforming LDP Paging server is a conforming LDP server [[!LDP]] that follows the rules defined by LDP Paging.

Example paging message exchanges

Example Co.'s customer relationship data is identified by the URI http://example.org/customer-relations, and is retrievable using the same URI.

Traditional flow without paging

A standard HTTP client that knows nothing about LDP paging obtains a representation of the resource in the usual way.

Request
GET /customer-relations HTTP/1.1
Host: example.org
Accept: text/turtle

The server response is:

Response:
HTTP/1.1 200 OK
Content-Type: text/turtle
ETag: "_87e52ce291112"
Link: <http://www.w3.org/ns/ldp#Resource>; rel="type"
Allow: GET,OPTIONS,HEAD
Transfer-Encoding: chunked

@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>.
@prefix dcterms: <http://purl.org/dc/terms/>.
@prefix foaf: <http://xmlns.com/foaf/0.1/>.
@prefix ldp: <http://www.w3.org/ns/ldp#>.
@prefix o: <http://example.org/ontology/>.
@base <http://example.org/customer-relations>.

<>
   a o:CustomerRelations;
   dcterms:title "The customer information for Example Co.";
   o:client <#JohnZSmith>, <#BettyASmith>, <#JoanRSmith>,
            <#GlenWSmith>, <#AlfredESmith>. 

<#JohnZSmith>
   a foaf:Person;
   o:status o:ActiveCustomer;
   foaf:name "John Z. Smith".
   
<#BettyASmith>
   a foaf:Person;
   o:status o:PreviousCustomer;
   foaf:name "Betty A. Smith".
 
 <#JoanRSmith>
   a foaf:Person;
   o:status o:PotentialCustomer;
   foaf:name "Joan R. Smith".
 
<#GlenWSmith>
   a foaf:Person;
   o:status o:ActiveCustomer, o:GoldCustomer;
   foaf:name "Glen W. Smith".

<#AlfredESmith>
   a foaf:Person;
   o:status o:ActiveCustomer, o:BronzeCustomer;
   foaf:name "Alfred E. Smith".

As long as the resource being retrieved is small enough that the server can form its representation without running into implementation limits on memory etc., and as long as the client is likewise able to consume these representations, this pattern works fine. Indeed, it's by far the most common pattern on the Web.

At the point where server and/or client constraints or consumption preferences come into play, however, something else is needed. LDP Paging addresses this need by giving clients and servers the ability to partition representations into smaller pieces, transfer as many as the client needs (potentially a small subset of the resource's full state), and allows the client to reassemble the pieces (or not) as it prefers. One sees this pattern in contexts such as search page results, as one common example.

Simple paging flow using redirects

The simplest possible solution would be to redefine the meaning of the existing URI http://example.org/customer-relations from "all customer relations information" to "the first chunk of all customer relations information". This would require changes to any existing clients whose code was built using the original definition however, so it's more likely that Example Co. would mint a new URI for "the first chunk", define a way to find subsequent chunks, and have clients use this new protocol.

The new protocol does not solve the problem of migrating existing clients from the old "all" to the new "first chunk" semantic; neither would a 303 See Other redirect from the old to the new URI, given the widespread historical client practice of automatically following 303 redirects and treating the redirected-to resource as equivalent to the original. The safe route is to have clients explicitly tell the server that they are capable of handling the "first chunk" semantic on the redirected-to URI; this is what LDP Paging does.

A client aware of LDP paging obtains a representation of the resource in the usual way, and also signals its ability to handle redirection to a first-page resource:

Request
GET /customer-relations HTTP/1.1
Host: example.org
Accept: text/turtle
Prefer: return=representation; page-size="500 rdf-triples"

The server's response contains a 303 See Other status code and a Location: http://example.org/customer-relations?page1 HTTP response header identifying the target resource, as shown below:

Response:
HTTP/1.1 303 See Other
Location: <http://example.org/customer-relations?page1>
Transfer-Encoding: chunked

At this point the client does not know if the target resource is "all" or "the first chunk"; it has to retrieve the resource to know.

Request
GET /customer-relations?page1 HTTP/1.1
Host: example.org
Accept: text/turtle
Prefer: return=representation; page-size="500 rdf-triples"

The server's response is shown below:

Response:
HTTP/1.1 200 OK
Content-Type: text/turtle
ETag: "_87e52ce291112"
Link: <http://www.w3.org/ns/ldp#Resource>; rel="type",
      <http://www.w3.org/ns/ldp#Page>; rel="type"
Link: <http://example.org/customer-relations?p=2>; rel='next'
Link: <http://example.org/customer-relations>; rel='canonical'; etag="customer-relations-v1"
Allow: GET,OPTIONS,HEAD
Transfer-Encoding: chunked

@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>.
@prefix dcterms: <http://purl.org/dc/terms/>.
@prefix foaf: <http://xmlns.com/foaf/0.1/>.
@prefix ldp: <http://www.w3.org/ns/ldp#>.
@prefix o: <http://example.org/ontology/>.
@base <http://example.org/customer-relations>.

<>
   a o:CustomerRelations;
   dcterms:title "The customer information for Example Co.";
   o:client <#JohnZSmith>, <#BettyASmith>, <#JoanRSmith>. 

<#JohnZSmith>
   a foaf:Person;
   o:status o:ActiveCustomer;
   foaf:name "John Z. Smith".
<#BettyASmith>
   a foaf:Person;
   o:status o:PreviousCustomer;
   foaf:name "Betty A. Smith".
 <#JoanRSmith>
   a foaf:Person;
   o:status o:PotentialCustomer;
   foaf:name "Joan R. Smith".

From this response, the client knows that more data exists and where to find it. The server determines the size of the pages using application-specific methods not defined within this specification, with the client's page-size preference as one input; the simplest method is to make the server's page size equal to the client's preference. In this example, the server chooses a smaller value so there is a second page.

The following example shows the message exchange for retrieving the next page:

Request
GET /customer-relations?p=2 HTTP/1.1
Host: example.org
Accept: text/turtle
Prefer: return=representation; page-size="500 rdf-triples"

The server's response is shown below:

Response:
HTTP/1.1 200 OK
Content-Type: text/turtle
ETag: "8_7e52ce291112"
Link: <http://www.w3.org/ns/ldp#Resource>; rel="type",
      <http://www.w3.org/ns/ldp#Page>; rel="type"
Link: <http://example.org/customer-relations>; rel='canonical'; etag="customer-relations-v1"
Allow: GET,OPTIONS,HEAD
Transfer-Encoding: chunked

@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>.
@prefix dcterms: <http://purl.org/dc/terms/>.
@prefix foaf: <http://xmlns.com/foaf/0.1/>.
@prefix ldp: <http://www.w3.org/ns/ldp#>.
@prefix o: <http://example.org/ontology/>.
@base <http://example.org/customer-relations>.

<>
   o:client <#GlenWSmith>, <#AlfredESmith>. 
 
<#GlenWSmith>
   a foaf:Person;
   o:status o:ActiveCustomer, o:GoldCustomer;
   foaf:name "Glen W. Smith".

<#AlfredESmith>
   a foaf:Person;
   o:status o:ActiveCustomer, o:BronzeCustomer;
   foaf:name "Alfred E. Smith".
 

In this example, there are only two customers provided in the second page.

Optimized paging flow

It is inconvenient that the 303 See Other approach requires three message exchanges to transfer two representations. Fortunately, it is possible to remove the extra message exchange, saving the latency and the server plus network overhead of servicing the 303 messages.

If a client signals that it is capable of understanding the HTTP status code 2NN Contents of Related, then the server can respond with 2NN Contents of Related instead of 303 See Other on the initial retrieval request, and it can also enclose the representation of the first page in the corresponding response.

Request
GET /customer-relations HTTP/1.1
Host: example.org
Accept: text/turtle
Prefer: return=representation; page-size="500 rdf-triples"
Prefer: contents-of-related

The server's response is shown below; it includes a status code 2NN Contents of Related, and a representation payload.

As was true in the 303 See Other approach, the server includes a Location: http://example.org/customer-relations?page1 HTTP response header identifying the first page as the resource whose representation is enclosed in the response.

Response:
HTTP/1.1 2NN Contents of Related
Content-Type: text/turtle
ETag: "_87e52ce291112"
Location: <http://example.org/customer-relations?page1>
Link: <http://www.w3.org/ns/ldp#Resource>; rel="type",
      <http://www.w3.org/ns/ldp#Page>; rel="type"
Link: <http://example.org/customer-relations?p=2>; rel='next'
Link: <http://example.org/customer-relations>; rel='canonical'; etag="customer-relations-v1"
Allow: GET,OPTIONS,HEAD
Transfer-Encoding: chunked

@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>.
@prefix dcterms: <http://purl.org/dc/terms/>.
@prefix foaf: <http://xmlns.com/foaf/0.1/>.
@prefix ldp: <http://www.w3.org/ns/ldp#>.
@prefix o: <http://example.org/ontology/>.
@base <http://example.org/customer-relations>.

<>
   a o:CustomerRelations;
   dcterms:title "The customer information for Example Co.";
   o:client <#JohnZSmith>, <#BettyASmith>, <#JoanRSmith>. 

<#JohnZSmith>
   a foaf:Person;
   o:status o:ActiveCustomer;
   foaf:name "John Z. Smith".
<#BettyASmith>
   a foaf:Person;
   o:status o:PreviousCustomer;
   foaf:name "Betty A. Smith".
 <#JoanRSmith>
   a foaf:Person;
   o:status o:PotentialCustomer;
   foaf:name "Joan R. Smith".

From this response, the client knows that it received the representation of an in-sequence page resource instead of the paged resource, that at least one more page exists, and where to find the next page. The server determines the size of the pages using application-specific methods not defined within this specification, with the client's page-size preference as one input; the simplest method is to make the server's page size equal to the client's preference. In this example, the server chooses a smaller value so there is a second page.

The following example shows the message exchange for retrieving the next page:

Request
GET /customer-relations?p=2 HTTP/1.1
Host: example.org
Accept: text/turtle
Prefer: return=representation; page-size="500 rdf-triples"
Prefer: contents-of-related

The server's response is shown below:

Response:
HTTP/1.1 200 OK
Content-Type: text/turtle
ETag: "8_7e52ce291112"
Link: <http://www.w3.org/ns/ldp#Resource>; rel="type",
      <http://www.w3.org/ns/ldp#Page>; rel="type"
Link: <http://example.org/customer-relations>; rel='canonical'; etag="customer-relations-v1"
Allow: GET,OPTIONS,HEAD
Transfer-Encoding: chunked

@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>.
@prefix dcterms: <http://purl.org/dc/terms/>.
@prefix foaf: <http://xmlns.com/foaf/0.1/>.
@prefix ldp: <http://www.w3.org/ns/ldp#>.
@prefix o: <http://example.org/ontology/>.
@base <http://example.org/customer-relations>.

<>
   o:client <#GlenWSmith>, <#AlfredESmith>. 
 
<#GlenWSmith>
   a foaf:Person;
   o:status o:ActiveCustomer, o:GoldCustomer;
   foaf:name "Glen W. Smith".

<#AlfredESmith>
   a foaf:Person;
   o:status o:ActiveCustomer, o:BronzeCustomer;
   foaf:name "Alfred E. Smith".
 

In this example, there are only two customers provided in the second page.

Linked Data Platform Paging Clients

All of the following are conformance rules for LDP Paging Clients.

General

LDP Paging clients MUST advertise their ability to support LDP Paging on all retrieval requests that normally result in a response containing a representation.

For Discussion

Given that servers Must Not initiate paging unless the client acks that it understands LDP paging, e.g. by sending a Prefer: pagesize header, do we want to add a Must on clients to send that header? Some support for doing so on the mailing list week of 7 July 2014 resulted in this strawman.

LDP Paging clients MUST be capable of at least one of forward traversal and/or backward traversal.

LDP Paging clients MUST NOT assume that any in-sequence page resource's paging links will remain unchanged when the in-sequence page resource is retrieved more than once. Such an assumption would conflict with a server's ability to add pages to a sequence as the paged resource changes, for example.

LDP Paging clients MUST NOT assume that any in-sequence page resource's paging links will always be accessible.
Non-normative note: Such an assumption would conflict with a server's ability to remove pages from a sequence as the paged resource changes, for example. One consequence of this is that clients can receive responses with 4xx status codes when following page links, purely due to timing issues and without any error on the part of the client or the server. Such a response would be unusual, and would likely signal an error, if the response also indicates that the paged resource's state has not changed while the client was traversing its pages.

LDP Paging clients SHOULD NOT present paged resources as coherent or complete, or make assumptions to that effect.

Client preferences

LDP Paging clients must provide paging-related hints in order to influence an LDP Paging server's choices.

This specification introduces new parameters on the HTTP Prefer request header's return=representation preference [[!RFC7240]], used by clients to supply a preference that helps the server form a response that is most appropriate to the client's needs. The preference serves several purposes:

Non-normative note: LDP server implementers should carefully consider the effects of these preferences on caching, as described in section 2 of [[!RFC7240]].
Non-normative note: [[!RFC7240]] recommends that server implementers include a Preference-Applied response header when the client cannot otherwise determine the server's behavior with respect to honoring hints from the response content.

LDP Paging defines page-size as a new parameter on the existing HTTP Prefer request header's return=representation preference [[!RFC7240]]. A client communicates its hint to the server by adding the request header with a return=representation preference that includes a page-size parameter whose value adheres to the following syntax:


		page-size-parameter = "page-size" *WSP "=" *WSP DQUOTE *WSP 1*DIGIT 1*WSP 1*units *WSP DQUOTE
		units               = rdf-triples / extension-units
		rdf-triples         = "rdf-triples"   ; units = RDF triples
		extension-units     = token           ; other units allowed for future extensibility
		

WSP is whitespace [[!RFC5234]], DIGIT is an integer between zero and nine [[!RFC5234]], DQUOTE is a double quote [[!RFC5234]], and token is as defined in [[!RFC7230]]. The generic preference BNF [[!RFC7240]] allows either a quoted string or a token as the value of a preference parameter; LDP Paging assigns a meaning to the value only when it is a quoted string. LDP Paging servers MAY ignore a page size of zero, or unrecognized units, and process the request as if no maximum desired size was specified; in the latter case the server can select whatever page size it deems appropriate, or choose not to page the resource at all.

Clients interested in receiving at most 500 RDF triples per page would use add this HTTP header on a GET request, as shown in : Prefer: return=representation; page-size="500 rdf-triples"

Linked Data Platform Resources

Introduction

Editorial note

There is more duplication between this content (its original home) and other sections (, ) than we'd prefer. It's on-going work to reduce the duplication.

Some LDPRs are too large to reasonably transmit a representation in a single HTTP response. To address this problem, LDP Paging servers can support a technique called Paging. When a client retrieves such a resource, the server redirects the client to a "first page" resource, and includes in its response a link to the next part of the resource's state, all at URLs of the server's choosing. The representation of each page of an LDPR is typically a subset of the paged resource. Any LDPR can be paged, but certain aspects of paging like ordering are only well-defined for LDP-RS's or LDPCs.

Logically, paging breaks up the paged resource into a list of in-sequence page resources (chunks) whose representations the client can retrieve. A server can offer links that support forward and/or backward traversal of the pages, but it has to offer at least one. Likewise, some clients will be interested in knowing whether or not competing requests altered the paged resource while the client was retrieving pages, since LDP paging does not guarantee that those alterations were reflected in any in-sequence page resource received by the client. Regardless of the server implementation, LDP only guarantees that while traversing a series of pages that the client observes data that is equivalent to a database that uses read committed isolation [[READ-COMMITTED]]; specifically, clients can observe non-repeatable reads [[NON-REPEATABLE-READS]] while traversing pages served by LDP Paging servers. LDP Paging does guarantee, however, that any information in the LDPR continuously present (neither added nor deleted) in the paged resource across the entire period of time when the client is retrieving pages will be present in at least one in-sequence page resource. LDP Paging defines a mechanism for informing clients that the paged resource has been changed so that they can, for example, abandon a page traversal as early as possible.

A list of in-sequence page resources is inherently ordered by the links that a LDP Paging server exposes on any paged resource. LDP Paging defines a vocabulary for communicating the ordering algorithm used by the LDP Paging server only in the case where the paged resource is an LDPC; other specifications or future versions of LDP Paging might define ordering in other cases.

Paging

Introduction

Editorial note

There is more duplication between this content (its original home) and other sections (, ) than we'd prefer. It's on-going work to reduce the duplication.

LDP Paging servers may respond to requests for a resource by redirecting to the first page of the resource and, with that, returning a next page link containing the URL for the next page. Clients inspect each response for the link to see if additional pages are available; paging does not affect the choice of HTTP status code. Note that paging provides a weak guarantee of completeness: if and only if it is the case that no changes to the paged resource are made while a client retrieves every in-sequence page resource logically comprising it, then the client will expect to have a complete picture of the paged resource at a point in time. Since LDP Paging ensures that clients have a way to find out when the paged resource has changed, this is a stronger guarantee than the one described for paged feeds [[RFC5005]], but it does require the client to detect when the condition holds. Paging does not guarantee that it will ever be possible to successfully retrieve all the in-sequence page resources without the paged resource being added to or changed, so clients requiring such a guarantee may not find all paged resources usable in practice. A detailed example is provided in . See for server implementation considerations.

HTTP GET

In addition to the requirements on HTTP GET for LDPRs [[!LDP]], LDP Paging servers must also follow the requirements in this section for all paged resources and their associated in-sequence page resources.

LDP Paging servers SHOULD allow clients to retrieve large LDP-RSs in pages.

LDP Paging servers MAY treat any resource (LDP-RS or not) as a paged resource.

LDP Paging servers MAY vary their treatment of any resource (LDP-RS or not) as a paged resource over time. In other words, given two attempts to retrieve the same resource at different points in time, the server can choose to return a representation of the first page at one time and of the entire resource at a different time. Clients distinguish between these cases based on the status code and response headers.

LDP Paging servers SHOULD respect all of a client's LDP-Paging-defined hints, for example the largest page size the client is interested in processing, to influence the amount of data returned in representations.

Feature At Risk

The LDP Working Group proposes incorporation of the features described in the next compliance clause.

  • A December 2013 TAG discussion started, whose goal is to reduce the need for two request-response round trips down to one when retrieving what turns out to be the first page of a paged resource, by defining a new HTTP response code in the 2xx or 3xx class that would allow a server to respond to GET request-uri requests with the representation of the first page (whose URI is first-page-uri, not request-uri) of a multi-page response.

  • For the purposes of drafting this section, we assume that the new code's value is 2NN Contents of Related, and its definition is given by the IETF draft [[!2NN]], whose content evolved from an LDP extrapolation from TAG discussions, Henry Thompson's strawman, with the substitution of 2NN for 2xx as suggested in the TAG's draft comments, item 2. Note: nothing prevents servers or clients from using 303 See Other responses to requests for paged resources. The only significant differences between 303 and 2NN responses are the extra round trip required for the client to retrieve the representation of the first page when using 303, and the non-cacheable nature of 303 responses.

  • Once LDP-Paging is a Candidate Recommendation, the LDP WG will make an assessment based on the status at IETF, working with the W3C Director, to either use the newly defined response code 2NN as documented in this section or to revert to a classic 303 response pattern.

LDP Paging servers SHOULD respond with HTTP status code 2NN Contents of Related to successful GET requests with any paged resource as the Request-URI when the request indicates the client's support for that status code [[!2NN]], although any appropriate code such as 303 See Other MAY be used.

LDP Paging servers MUST ensure that all state present in the paged resource throughout a client's entire traversal operation is represented in at least one in-sequence page resource. In other words, whatever subset of the paged resource that is not added, updated, or removed during the client's traversal of its pages has to be present in one of the pages.

Non-normative note: As a consequence, if the paged resource does not change at all during the traversal, then the client has a complete view of its state as given by the negotiated response media type at the point in time when the final page was retrieved. If the paged resource does change during the traversal, then only the portions that were actually updated will differ; the client has no LDP Paging-provided means for knowing in what way(s) its view differs from that of the server in this case.
Non-normative note: When the paged resource is an LDP-RS, the expectation is that all triples untouched by changes to the paged resource have been given to the client during the traversal; it is possible that some subset of the changed triples, including all or none of them, have been provided to the client, but the client has no way to know which.

LDP Paging servers MUST enable a client to detect any change to the paged resource that occurs while the client is retrieving pages by including a HTTP Link header on all successful HTTP GET responses, and SHOULD include the same header on all 4xx status code responses. The link's context URI identifies the in-sequence page resource being retrieved, target URI identifies the paged resource, link relation type is canonical [[!REL-CANONICAL]], and link extension parameters include the parameter name etag and a corresponding parameter value identical to the ETag [[!RFC7232]] of the paged resource. For example: Link: <http://example.org/customer-relations>; rel='canonical'; etag="customer-relations-v1"
Non-normative note: If the rel='canonical'; etag="..." value changes as the client retrieves pages, for example the value accompanying the first page's representation is rel='canonical'; etag="v1" and the value accompanying the second page's representation is rel='canonical'; etag="v2", the client can detect that the paged resource's state has changed. Some clients will ignore such changes, but others may choose to react to them, for example by restarting the traversal.

LDP Paging servers MAY add or remove in-sequence page resources to a paged resource's sequence over time, but SHOULD only add pages to the end of a sequence.

Non-normative note: As a result, clients retrieving any in-sequence page resource several times can observe its place in the sequence change as the state of the paged resource changes. For example, a nominally last page's server might provide a next page link when the page is retrieved again later. Similar situations arise when the page sequence crosses server boundaries; server A might host the initial portion of a sequence that links to the last page server A is aware of, hosted on server B, and server B might extend the sequence of pages. A nominally middle page M can become the last (or a non-existent) page if a competing request deletes part of the paged resource's content after the client retrieves M.

LDP Paging servers MAY provide a first page link when responding to requests with any in-sequence page resource as the Request-URI.

LDP Paging servers MAY provide a last page link in responses to GET requests with any in-sequence page resource as the Request-URI.

LDP Paging servers MUST provide a next page link in responses to GET requests with any in-sequence page resource other than the final page as the Request-URI. This is the mechanism by which clients can discover the URL of the next page.

LDP Paging servers MUST NOT provide a next page link in responses to GET requests with the final in-sequence page resource as the Request-URI. This is the mechanism by which clients can discover the end of the page sequence as currently known by the server.

LDP Paging servers MAY provide a previous page link in responses to GET requests with any in-sequence page resource other than the first page as the Request-URI. This is one mechanism by which clients can discover the URL of the previous page.

LDP Paging servers MUST NOT provide a previous page link in responses to GET requests with the first in-sequence page resource as the Request-URI. This is one mechanism by which clients can discover the beginning of the page sequence as currently known by the server.

LDP Paging servers MUST provide an HTTP Link header whose target URI is http://www.w3.org/ns/ldp#Page, and whose link relation type is type [[!RFC5988]] in responses to GET requests with any in-sequence page resource as the Request-URI. This is one mechanism by which clients know that the resource is one of a sequence of pages.

For Discussion

Need to clarify expected behavior(s). Some support for doing so on the mailing list week of 7 July 2014.

Strawman based on Sandro's response: add to the normative requirement below:

Non-normative note: LDP Paging servers could choose to make any resource available only as a paged resource. One consequence of the prohibition on initiating paging when interacting with non-paging-aware clients is: if such a server receives a request for a paged-only resource, and the request does not signal that the client is paging-aware, then the server has to reject the request, most likely with a 4xx status code. This avoids the situation where a non-paging-aware client blindly follows a 303 redirect, retrieves that resource (which the server, but not the client, knows to contain only the first page of the paged resource's state), and upon receiving the 200 OK status code concludes that it now has the entire representation of the paged resource's state (instead of only having a representation of the subset assigned to the first page).

LDP Paging servers MUST NOT initiate paging unless the client has indicated it understands paging. The only standard means defined by LDP paging for a client to signal a server that the client understands paging is via the client preference defined for this purpose; other implementation-specific means could also be used.

Linked Data Platform Containers

Introduction

Many HTTP applications and sites have organizing concepts that partition the overall space of resources into smaller containers. Blog posts are grouped into blogs, wiki pages are grouped into wikis, and products are grouped into catalogs. Each resource created in the application or site is created within an instance of one of these container-like entities, and users can list the existing artifacts within one. LDP Paging Containers answer some basic questions, which are:

  1. How is the order of the container entries expressed?

Ordering

There are many cases where an ordering of the members of a container is important. LDP does not provide any particular support for server ordering of members in containers, because any client can order the members in any way it chooses based on the value of any available property of the members. In the example below, the value of the o:value predicate is present for each member, so the client can easily order the members according to the value of that property. In this way, LDP avoids the use of RDF constructs like Seq and List for expressing order.

Order becomes more important when containers are paginated. If the server respects ordering when constructing pages, clients needing a globally sorted set of members can reduce the effort required to merge pages. In cases where ordering is important, an LDP Paging server exposes all the members on a page with the proper sort order with relation to all members on the next and previous pages. When the sort is ascending, all the members on a current page have a sort order no lower than all members on the previous page and sort order no higher than all the members on the next page; that is, it proceeds from low to high, but keep in mind that several consecutive pages might have members whose sort criteria are equal. When the sort is descending, the opposite order is used. Since more than one value may be used to sort members, the LDPC specification allows servers to assert the ordered list of sort criteria used to sort the members, using the ldp:containerSortCriteria relation. Each member of the ordered list exposes one ldp:containerSortCriterion, consisting of a ldp:containerSortOrder, ldp:containerSortPredicate, and optionally a ldp:containerSortCollation.

Here is an example asset container described in [[LDP]] section 5.1, with the ordering of the assets asserted:

Request
GET /netWorth/nw1/assetContainer/ HTTP/1.1
Host: example.org
Accept: text/turtle

The server responds with status code 200 OK, and the following representation:

Response:
HTTP/1.1 200 OK
Content-Type: text/turtle
ETag: "_87e52ff291112"
Link: <http://www.w3.org/ns/ldp#Resource>; rel="type"
Allow: GET,OPTIONS,HEAD
Transfer-Encoding: chunked

# The following is the ordered representation of
#   http://example.org/netWorth/nw1/assetContainer/

# @base <http://example.org/netWorth/nw1/assetContainer/>
@prefix dcterms: <http://purl.org/dc/terms/>.
@prefix ldp: <http://www.w3.org/ns/ldp#>.
@prefix o: <http://example.org/ontology/>.

<>
   a ldp:DirectContainer;
   dcterms:title "The assets of JohnZSmith";
   ldp:membershipResource <http://example.org/netWorth/nw1>;
   ldp:hasMemberRelation o:asset;
   ldp:insertedContentRelation ldp:MemberSubject.

<?firstPage>
   a ldp:Page;
   ldp:pageOf <>;
   ldp:containerSortCriteria (<#SortValueAscending>).

<#SortValueAscending>
   a ldp:ContainerSortCriterion;
   ldp:containerSortOrder ldp:Ascending;
   ldp:containerSortPredicate o:value.

<http://example.org/netWorth/nw1>
   a o:NetWorth;
   o:asset <a1>, <a3>, <a2>.

<a1>
   a o:Stock;
   o:value 100.00 .
<a2>
   a o:Cash;
   o:value 50.00 .
<a3>
   a o:RealEstateHolding;
   o:value 300000 .

As you can see by the addition of the ldp:ContainerSortCriteria predicate, the o:value predicate is used to order the page members in ascending order. It is up to the domain model and server to determine the appropriate predicate to indicate the resource’s order within a page, and up to the client receiving this representation to use that order in whatever way is appropriate, for example to sort the data prior to presentation on a user interface. Also as it is possible for a container to have as its members other containers, the ordering approach doesn't change as containers themselves are LDPRs and the properties from the domain model can be leveraged for the sort criteria.

General

The Linked Data Platform does not define how clients discover LDPCs.

For Discussion

Are we attempting to define a new type of container here? If not, should delete the clause. Some support for doing so on the mailing list week of 7 July 2014.

Each Linked Data Platform Paging Container MUST also be a conforming Linked Data Platform Container.

LDP Paging servers MUST ensure that the membership triple and containment triple for each member are part of the same in-sequence page resource, whenever both triples are present in the page sequence for a paged LDPC.

HTTP GET

LDP Paging servers MAY communicate the order it uses to allocate LDPC members to in-sequence page resources as part of the pages' representations; LDP Paging does not specify ordering for pages of LDPRs in other cases. If the server communicates this, it MUST specify the order using a triple whose subject is the page URI, whose predicate is ldp:containerSortCriteria, and whose object is a rdf:List of ldp:containerSortCriterion resources. The resulting order MUST be as defined by SPARQL SELECT’s ORDER BY clause [[!sparql11-query]]. Sorting criteria MUST be the same for all pages of a representation; if the criteria were allowed to vary, the ordering among members of a container across pages would be undefined. The first list entry provides the primary sorting criterion, any second entry provides a secondary criterion used to order members considered equal according to the primary criterion, and so on. See for an example.

LDPC page representations ordered using ldp:containerSortCriteria MUST contain, in every ldp:containerSortCriterion list entry, a triple whose subject is the sort criterion identifier, whose predicate is ldp:containerSortPredicate and whose object is the predicate whose value is used to order members between pages (the page-ordering values). The only literal data types whose behavior LDP constrains are those defined by SPARQL SELECT’s ORDER BY clause [[!sparql11-query]]. Other data types can be used, but LDP assigns no meaning to them and interoperability will be limited.

LDPC page representations ordered using ldp:containerSortCriteria MUST contain, in every ldp:containerSortCriterion list entry, a triple whose subject is the sort criterion identifier, whose predicate is ldp:containerSortOrder and whose object describes the order used. LDP defines two values, ldp:Ascending and ldp:Descending, for use as the object of this triple. Other values can be used, but LDP assigns no meaning to them and interoperability will be limited.

LDPC page representations ordered using ldp:containerSortCriteria MAY contain, in any ldp:containerSortCriterion list entry, a triple whose subject is the sort criterion identifier, whose predicate is ldp:containerSortCollation and whose object identifies the collation used. LDP defines no values for use as the object of this triple. While it is better for interoperability to use open standardized values, any value can be used. When the ldp:containerSortCollation triple is absent and the page-ordering values are strings or simple literals [[!sparql11-query]], the resulting order is as defined by SPARQL SELECT’s ORDER BY clause [[!sparql11-query]] using two-argument fn:compare, that is, it behaves as if http://www.w3.org/2005/xpath-functions/collation/codepoint was the specified collation. When the ldp:containerSortCollation triple is present and the page-ordering values are strings or simple literals [[!sparql11-query]], the resulting order is as defined by SPARQL SELECT’s ORDER BY clause [[!sparql11-query]] using three-argument fn:compare, that is, the specified collation. The ldp:containerSortCollation triple MUST be omitted for comparisons involving page-ordering values for which [[!sparql11-query]] does not use collations.

Security Considerations

As with any protocol that is implemented leveraging HTTP, implementations should take advantage of the many security-related facilities associated with it and are not required to carry out LDP operations that may be in contradistinction to a particular security policy in place. For example, when faced with an unauthenticated request to replace system critical RDF statements in a graph through the PUT method, applications may consider responding with the 401 status code (Unauthorized), indicating that the appropriate authorization is required. In cases where authentication is provided fails to meet the requirements of a particular access control policy, the 403 status code (Forbidden) can be sent back to the client to indicate this failure to meet the access control policy.

Paging LDPRs without maintaining server-side session state

Server implementers naturally have concerns when they are expected to maintain per-client state because of the scalability limitations that per-client state implies. LDP Paging carries no such requirement, although this may not be obvious at first glance. Since URLs are opaque to clients [[WEBARCH]], a server is free to encode the information required for it to know where to start the next page inside its next page links, for example.

Observe that in the preceding examples, the page n URIs are all of the form http://example.org/customer-relations?p=n, where n is 2..n. This is likely true in general practice. If the server allocates o:client representations to pages randomly, it's not obvious how to avoid keeping per-client state of one kind or another on the server while still sending all the representations on at least one page. In the common case where the list has an ordering (either a natural one or one imposed by the implementation) however, it is easy to avoid keeping per-client state on the server.

One trivial case is "fixed order"; if the server always sends o:client representations in the order #JohnZSmith, #BettyASmith, #JoanRSmith, #GlenWSmith, #AlfredESmith (or indeed, in any predictable order), then it can put the "last seen" identifier in the next page link of each in-sequence page resource as it is retrieved by each client. The "session state" is, in effect, kept by the client. In this case, the next page link in the first page might be:

		Link: <http://example.org/customer-relations?resumeafter=JoanRSmith>; rel='next'
	

If the server also supports backward traversal, then the second page's previous page link might be:

		Link: <http://example.org/customer-relations?resumebefore=GlenWSmith>; rel='next'
	

Keep in mind that this is an exemplary server implementation decision; it is not prescribed by LDP Paging, and other choices are certainly possible. As long as the URIs used in links are opaque to clients, any choice within the constraints of all normative references is permissible.

Acknowledgements

The following people have been instrumental in providing thoughts, feedback, reviews, content, criticism and input in the creation of this specification:

Alexandre Bertails, Andrei Sambra, Andy Seaborne, Antonis Loizou, Arnaud Le Hors, Ashok Malhota, Bart van Leeuwen, Cody Burleson, David Wood, Eric Prud'hommeaux, Erik Wilde, Henry Story, John Arwe, Kevin Page, Kingsley Idehen, Mark Baker, Martin P. Nally, Miel Vander Sande, Miguel Esteban Gutiérrez, Nandana Mihindukulasooriya, Olivier Berger, Pierre-Antoine Champin, Raúl García Castro, Reza B'Far, Richard Cyganiak, Roger Menday, Ruben Verborgh, Sandro Hawke, Serena Villata, Sergio Fernandez, Steve Battle, Steve Speicher, Ted Thibodeau, Tim Berners-Lee, Yves Lafon

Change History

The change history is up to the editors to insert a brief summary of changes, ordered by most recent changes first and with heading from which public draft it has been changed from.

February 18, 2014 Editor's Draft