Added instructions for running tests.
authorGregg Kellogg <gregg@kellogg-assoc.com>
Fri, 27 Dec 2013 15:19:20 -0800
changeset 1699 af637bccf6ac
parent 1698 e5e8fd218cf7
child 1700 6c30736ab59b
Added instructions for running tests.
local-biblio.js
rdf11-testcases/index.html
--- a/local-biblio.js	Fri Dec 27 16:13:01 2013 +0100
+++ b/local-biblio.js	Fri Dec 27 15:19:20 2013 -0800
@@ -36,6 +36,7 @@
     "SPARQL11-OVERVIEW": { aliasOf: "sparql11-overview" },
     "SPARQL11-ENTAILMENT": { aliasOf: "sparql11-entailment" },
     "SPARQL11-SERVICE-DESCRIPTION": { aliasOf: "sparql11-service-description" },
+    "SPARQL11-TEST-CASE" : "Axel Polleres. <cite><a href=\"http://www.w3.org/2009/sparql/docs/tests/README.html\">SPARQL1.1: Test case structure</a></cite> Informal. URL: <a href=\"http://www.w3.org/2009/sparql/docs/tests/README.html\">http://www.w3.org/2009/sparql/docs/tests/README.html</a>",
     "SPARQL11-UPDATE": { aliasOf: "sparql11-update" },
     "XPATH-DATAMODEL-30": { aliasOf: "xpath-datamodel-30" },
     "XPATH-FUNCTIONS-30": { aliasOf: "xpath-functions-30" },
--- a/rdf11-testcases/index.html	Fri Dec 27 16:13:01 2013 +0100
+++ b/rdf11-testcases/index.html	Fri Dec 27 15:19:20 2013 -0800
@@ -1,7 +1,7 @@
 <!DOCTYPE html>
 <html lang="en">
   <head>
-    <meta charset="utf-8">
+    <meta charset="utf-8"/>
     <title>RDF 1.1 Test Cases</title>
     <script src="../local-biblio.js" class="remove"></script>
     <script src="https://www.w3.org/Tools/respec/respec-w3c-common" class="remove"></script>
@@ -69,13 +69,27 @@
           // from the dcterms, foaf, and bibo. The parameter defaults to false.
           doRDFa: "1.1",
 
-          noTOC:  true,
+          noTOC:  false,
 
           // alternateFormats: [ { uri: "diff-20130723.html", label: "diff to previous version" } ]
       };
     </script>
+    <style type="text/css">
+    /* Style Turtle script blocks to be visable */
+    pre.example script {
+        display:block;
+    }
+    .separated { border-collapse:collapse; }
+    .separated thead tr th { border:1px solid black; padding: .2em; }
+    .separated tbody tr td { border:1px solid black; text-align: center; }
+    .separated tbody tr td.r { text-align: right; padding: .5em; }
+    .grammar td { font-family: monospace; vertical-align: top; }
+    .grammar-literal { color: gray;}
+	.grammar_comment { color: #A52A2A; font-style: italic; }
+    </style> 
   </head>
 <body>
+  <section id='sotd'></section>
   <section id="abstract">
     <p>The Resource Description Framework (RDF) is a framework for
       representing information in the Web. This document lists
@@ -85,26 +99,169 @@
   </section>
 
   <section>
-    <h1>Test Suites and Implementation Reports</h1>
+    <h2>General instructions for running RDF Test suites</h2>
+    <p>All test suites have a common test manifest representation using test vocabularies
+      similar to the SPARQL 1.1 Test case structure [[SPARQL11-TEST-CASE]]. Test manifests
+      are expressed using [[TURTLE]], except for the JSON-LD test manifests, which
+      are represented using [[JSON-LD]]. (Detailed information on the JSON-LD Test Suite
+      is at [[JSON-LD-TESTS]]). The Manifest has a header, and lists one
+      or more test entries such as the following:</p>
+    <pre class="example" title="Manifest header"><script type="text/turtle">&lt;&gt;  rdf:type mf:Manifest ;
+   rdfs:comment &quot;Turtle tests&quot; ;
+   mf:entries
+   (
+   ....
+   ) .</script></pre>
+
+    <p>In addition to the namespaces described in [[SPARQL11-TEST-CASE]], RDF manifests use
+      the following vocabularies:</p>
+    <dl>
+      <dt><code>rdft</code></dt>
+      <dd><code>http://www.w3.org/ns/rdftest#</code></dd>
+    </dl>
+
+    <section>
+      <h3>Syntax Tests</h3>
+      <p>Concrete RDF syntaxes may have positive- and negative-syntax tests, intended
+        to determine if a processor correctly recognizes valid or invalid serializations.
+        Positive Syntax Tests include a <em>name</em>, optional <em>comment</em>, and <em>action</em> along
+        with a type which identifies them as being a Syntax Test for the appropriate
+        serialization format. An example Turtle Syntax test is the following:</p>
+
+      <pre class="example" title="Syntax test"><script type="text/turtle">&lt;#turtle-syntax-file-01&gt; rdf:type rdft:TestTurtlePositiveSyntax ;
+   mf:name    &quot;turtle-syntax-file-01&quot; ;
+   rdfs:comment &quot;Empty file&quot; ;
+   mf:action    &lt;turtle-syntax-file-01.ttl&gt; ;
+   .</script></pre>
+
+     <p>Class names vary by serialization format and include the following:</p>
+     <dl>
+       <dt><code>rdft:TestTurtlePositiveSyntax</code></dt><dd>A Positive [[TURTLE]] Syntax Test.</dd>
+       <dt><code>rdft:TestTurtleNegativeSyntax</code></dt><dd>A Negative [[TURTLE]] Syntax Test.</dd>
+       <dt><code>rdft:TestTriGPositiveSyntax</code></dt><dd>A Positive [[TRIG]] Syntax Test.</dd>
+       <dt><code>rdft:TestTriGNegativeSyntax</code></dt><dd>A Negative [[TRIG]] Syntax Test.</dd>
+       <dt><code>rdft:TestNTriplesPositiveSyntax</code></dt><dd>A Positive [[N-TRIPLES]] Syntax Test.</dd>
+       <dt><code>rdft:TestNTriplesNegativeSyntax</code></dt><dd>A Negative [[N-TRIPLES]] Syntax Test.</dd>
+       <dt><code>rdft:TestNQuadsPositiveSyntax</code></dt><dd>A Positive [[N-QUADS]] Syntax Test.</dd>
+       <dt><code>rdft:TestNQuadsNegativeSyntax</code></dt><dd>A Negative [[N-QUADS]] Syntax Test.</dd>
+     </dl>
+
+     <p>A positive test is passed if an implementation can parse the content at the URL identified
+       by the object of the <code>mf:action</code> property using rules for the the appropriate syntax.</p>
+     <p>A negative test is passed if an implementation raises an error when it parses
+        content at the URL identified by the object of the <code>mf:action</code> property using rules for the appropriate syntax.</p>
+   </section>
+   <section>
+     <h3>Evaluation Tests</h3>
+     <p>Concrete RDF syntaxes may have positive- and negative-evaluation tests, intended
+       to determine if a processor correctly evaluates serializations.
+       Positive Evaluation Tests include a <em>name</em>, optional <em>comment</em>, <em>action</em>, and <em>result</em> along
+       with a type which identifies them as being an Evaluation Test for the appropriate
+       serialization format. An example Turtle Evaluation test is the following:</p>
+
+     <pre class="example" title="Evaluation test"><script type="text/turtle">&lt;#IRI_subject&gt; rdf:type rdft:TestTurtleEval ;
+   mf:name      &quot;IRI_subject&quot; ;
+   rdfs:comment &quot;IRI subject&quot; ;
+   mf:action    &lt;IRI_subject.ttl&gt; ;
+   mf:result    &lt;IRI_spo.nt&gt; ;
+   .</script></pre>
+
+     <p>Class names vary by serialization format and include the following:</p>
+     <dl>
+       <dt><code>rdft:TestTurtleEval</code></dt><dd>A Positive [[TURTLE]] Evaluation Test.</dd>
+       <dt><code>rdft:TestTurtleNegativeEval</code></dt><dd>A Negative [[TURTLE]] Evaluation Test.</dd>
+       <dt><code>rdft:TestTrigEval</code></dt><dd>A Positive [[TRIG]] Evaluation Test.</dd>
+       <dt><code>rdft:TestTrigNegativeEval</code></dt><dd>A Negative [[TRIG]] Evaluation Test.</dd>
+     </dl>
+
+     <p>A positive test is passed if an implementation parses the input
+       (identified by the object of the <code>mf:action</code> property) using the
+       appropriate syntax into a graph
+       or dataset, parses the expceted result
+       (identified by the object of the <code>mf:result</code> property)
+       into another graph or dataset, and those two graphs/datasets are
+       <a href="http://www.w3.org/TR/rdf11-concepts/#graph-isomorphism">isomorphic</a> (see [[RDF11-CONCEPTS]]).</p>
+       <p>A negative test is passed if an implementation parses the input
+         (identified by the object of the <code>mf:action</code> property) using the
+         appropriate syntax into a graph
+         or dataset, parses the expceted result
+         (identified by the object of the <code>mf:result</code> property)
+         into another graph or dataset, and those two graphs/datasets are _not_
+         <a href="http://www.w3.org/TR/rdf11-concepts/#graph-isomorphism">isomorphic</a> (see [[RDF11-CONCEPTS]]).</p>
+    </section>
+
+    <section>
+      <h3>Sematics Tests</h3>
+      <p>These tests test implementations of RDF and RDFS Entailment [[RDF11-MT]].
+        Entailment Tests (<code>mf:PositiveEntailmentTest</code> or <code>mf:NegativeEntailmentTest</code>) include a <em>name</em>, <em>comment</em>, <em>action</em>, <em>result</em>, an <em>entailment</em> regime,
+        a <em>list of recognized datatypes</em> and a <em>list of unrecognized datatypes</em>, and <em>result</em> along
+        with a type which identifies them as being an Entailment Test for the appropriate
+        serialization format.</p>
+      <p>All tests have the following properties:</p>
+      <ul>
+        <li>a name (<code>mf:name</code>)</li>
+        <li>an input RDF graph url (<code>mf:action</code>)</li>
+        <li>an output RDF graph URL or the special marker <code>false</code> (<code>mf:result</code>),</li>
+        <li>an entailment regime, which is "simple", "RDF", or "RDFS" (<code>mf:entailmentRegime</code>),</li>
+        <li>a list of recognized datatypes (<code>mf:recognizedDatatypes</code>),</li>
+        <li>a list of unrecognized datatypes (<code>mf:unrecognizedDatatypes</code>).</li>
+      </ul>
+      <p>An example Entailment test is the following:</p>
+
+     <pre class="example" title="Entailment test">&lt;#datatypes-non-well-formed-literal-2&gt; a mf:PositiveEntailmentTest;
+  mf:name &quot;datatypes-non-well-formed-literal-2&quot;;
+  rdfs:comment &quot;&quot;&quot;
+    With appropriate datatype knowledge, a &#x27;badly-formed&#x27;
+    datatyped literal can be detected.
+  &quot;&quot;&quot;;
+  rdfs:comment &quot;&quot;&quot;
+    Ill-formed datatyped literals now are inconsistent.
+    Used to be negative entailment to &lt;datatypes/test002b.nt&gt;
+  &quot;&quot;&quot;;
+  rdfs:approval rdft:Approved;
+  mf:entailmentRegime &quot;RDFS&quot; ;
+  mf:recognizedDatatypes ( xsd:integer ) ;
+  mf:unrecognizedDatatypes ( ) ;
+  mf:action &lt;datatypes/test002.nt&gt;;
+  mf:result false.</script></pre>
+
+    <p>An implementation passes a Positive (Negative) Entailment Test if, when
+    configured to
+     (1) perform entailment under the entailment regime of the test or some entailment regime that is stronger (weaker) than the entailment regime and
+     (2) recognize all the datatypes in the list of <em>recognized datatypes</em> and none of the datatypes in the list of <em>unrecognized datatypes</em>,</p>
+    <ul>
+    <li>for tests that have an output graph, determines that the input RDF graph entails (does not entail) the output RDF graph</li>
+    <li>for tests that have <code>false</code> as output, either determines that the input RDF graph entails (does not entail) an inconsistent RDF graph or that the input RDF graph is inconsistent (consistent).</li>
+  </ul>
+
+  <p>An implementation also passes a test if when configured differently from a
+    correct configuration as given above nonetheless produces the given result,
+    and the result is correct in the configured entailment regime with the
+    configured recognized datatypes.</p>
+
+    </section>
+  </section>
 
   <section>
-    <h2>RDF 1.1 Semantics [[RDF11-MT]]</h2>
-    
+    <h2>Test Suites and Implementation Reports</h2>
+    <p>The following are links to the syntax and semantics test suites and
+      implementation reports:</p>
+
     <ul>
-      <li><strong>RDF 1.1 Semantics</strong> [[RDF11-MT]]<br>
+      <li><strong>RDF 1.1 Semantics</strong> [[RDF11-MT]]<br/>
         <a href="http://www.w3.org/2013/rdf-mt-tests/">http://www.w3.org/2013/rdf-mt-tests/</a>
         (<a href="https://dvcs.w3.org/hg/rdf/raw-file/default/rdf-mt/reports/index.html">Implementation Report</a>)
       </li>
-      <li><p><strong>RDF 1.1 Turtle</strong> [[TURTLE]]<br>
-        <a href="http://www.w3.org/2013/TurtleTests/">http://www.w3.org/2013/TurtleTests/</a></li></p>
-      <li><p><strong>RDF 1.1 TriG</strong> [[TRIG]]<br>
-        <a href="http://www.w3.org/2013/TrigTests/">http://www.w3.org/2013/TrigTests/</a></li></p>
-      <li><p><strong>RDF 1.1 N-Triples</strong> [[N-TRIPLES]]<br>
-        <a href="http://www.w3.org/2013/N-TriplesTests/">http://www.w3.org/2013/N-TriplesTests/</a></li></p>
-      <li><p><strong>RDF 1.1 N-Quads</strong> [[N-QUADS]]<br>
-        <a href="http://www.w3.org/2013/N-QuadsTests/">http://www.w3.org/2013/N-QuadsTests/</a></li></p>
-      <li><p><strong>JSON-LD 1.0</strong> [[JSON-LD]]/[[JSON-LD-API]]<br>
-        <a href="http://www.w3.org/2013/json-ld-tests/">http://www.w3.org/2013/json-ld-tests/</a></li></p>
+      <li><strong>RDF 1.1 Turtle</strong> [[TURTLE]]<br/>
+        <a href="http://www.w3.org/2013/TurtleTests/">http://www.w3.org/2013/TurtleTests/</a></li>
+      <li><strong>RDF 1.1 TriG</strong> [[TRIG]]<br/>
+        <a href="http://www.w3.org/2013/TrigTests/">http://www.w3.org/2013/TrigTests/</a></li>
+      <li><strong>RDF 1.1 N-Triples</strong> [[N-TRIPLES]]<br/>
+        <a href="http://www.w3.org/2013/N-TriplesTests/">http://www.w3.org/2013/N-TriplesTests/</a></li>
+      <li><strong>RDF 1.1 N-Quads</strong> [[N-QUADS]]<br/>
+        <a href="http://www.w3.org/2013/N-QuadsTests/">http://www.w3.org/2013/N-QuadsTests/</a></li>
+      <li><strong>JSON-LD 1.0</strong> [[JSON-LD]]/[[JSON-LD-API]]<br/>
+        <a href="http://www.w3.org/2013/json-ld-tests/">http://www.w3.org/2013/json-ld-tests/</a></li>
     </ul>
   </section>
 </body>