--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/testcases/constraints/process.html Thu Oct 11 16:05:27 2012 +0100
@@ -0,0 +1,852 @@
+<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+<html xmlns="http://www.w3.org/1999/xhtml">
+<head>
+<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
+<title>PROV-CONTRAINT Test Cases</title>
+</head>
+
+<body>
+<h1>PROV-CONTRAINT Test Cases</h1>
+<p>Process for running and reporting the test results</p>
+
+<h2>Process for Implementers</h2>
+
+<h3>Test cases</h3>
+<p>The test cases will be made available online at: <URI><br />
+ Every test case has an identifier, e.g. <strong>entity10</strong>. The provenance document for the test case will be provided in the following representations:</p>
+
+<table border="1" cellspacing="0" cellpadding="0" width="623">
+ <caption>Table 1. File representations provided for test cases.</caption>
+ <tr>
+ <td><br />
+ <strong>Representation</strong></td>
+ <td><p><strong>File extension</strong></p></td>
+ <td><p><strong>Documentation</strong></p></td>
+ </tr>
+ <tr>
+ <td><p><strong>PROV-XML</strong></p></td>
+ <td><p>.xml</p></td>
+ <td><p><a href="http://dvcs.w3.org/hg/prov/raw-file/default/xml/prov-xml.html">http://dvcs.w3.org/hg/prov/raw-file/default/xml/prov-xml.html</a></p></td>
+ </tr>
+ <tr>
+ <td><p><strong>PROV-RDF</strong></p></td>
+ <td><p>.trig</p></td>
+ <td><p><a href="http://www.w3.org/TR/prov-o/">http://www.w3.org/TR/prov-o/</a></p></td>
+ </tr>
+ <tr>
+ <td><p><strong>PROV-JSON</strong></p></td>
+ <td><p>.json</p></td>
+ <td><p><a href="http://users.ecs.soton.ac.uk/tdh/json/">http://users.ecs.soton.ac.uk/tdh/json/</a></p></td>
+ </tr>
+ <tr>
+ <td><p><strong>PROV-N</strong></p></td>
+ <td><p>.provn</p></td>
+ <td><p><a href="http://www.w3.org/TR/prov-n/">http://www.w3.org/TR/prov-n/</a></p></td>
+ </tr>
+</table>
+
+<p>For example, the files <strong>entity10.xml</strong>, <strong>entity10.trig</strong>, <strong>entity10.json</strong>, and <strong>entity10.provn</strong> for the test case <strong>entity10</strong> are made available at <URI>.</p>
+
+<p>Each test case is designed to test particular constraints, an overview of which is provided in Table 2 below.</p>
+
+<h3>Test Results</h3>
+
+<p>Implementers are asked to validate each test case with their implementation and recorded the validation results <em>two plain text files</em>: <strong>success.txt</strong> and <strong>fail.txt</strong>. These files should contain the identifiers (one on each line) of the test cases that have been successfully validated in the former and those that fail validation in the latter. Test cases that are not supported by the implementation <strong>should not</strong> be included in either of the files.</p>
+
+<p>The test result files will need to be <emailed to|uploaded at>. </p>
+
+<p>Dong: What else the implementer should provide us with? With the test results, I think they won't need to fill in the questionnaire. Therefore, we
+might need to ask them to provide some basic information about their
+implementations (i.e. the questionnaire 1<sup>st</sup> page).</p>
+
+<h2>Process for the Working Group</h2>
+
+<h3>Categorizing Test Cases</h3>
+
+<p>All test cases need to be checked for the constraints they cover and their expected validation results (i.e. Success or Fail).</p>
+
+<p>Dong: These test cases are from Luc’s ProvToolbox. We need to check whether all the constraints are sufficiently covered and curate from the WG or generate ourselves the test cases for constraints that haven’t been covered by these.</p>
+
+<table border="1" cellspacing="0" cellpadding="0">
+ <caption>Table 2. Available test cases (127 in total)</caption>
+ <tr>
+ <td><b>Identifier</b></p></td>
+ <td><b>Constraints Tested</b></p></td>
+ <td><b>Expected Validation Result</b></p></td>
+ <td><b>Notes</b></p></td>
+ </tr>
+ <tr>
+ <td><b>activity1</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>activity2</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>activity3</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>activity4</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>activity5</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>activity6</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>activity7</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>activity8</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>activity9</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>agent1</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>agent2</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>agent3</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>agent4</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>agent5</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>agent6</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>agent7</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>agent8</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>alternate1</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>association1</b></p></td>
+ <td> </td>
+ <td>Fail</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>association2</b></p></td>
+ <td> </td>
+ <td>Fail</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>association3</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>association4</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>association5</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>association6</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>association7</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>association8</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>association9</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>attribution1</b></p></td>
+ <td> </td>
+ <td>Fail</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>attribution2</b></p></td>
+ <td> </td>
+ <td>Fail</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>attribution3</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>attribution4</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>attribution5</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>attribution6</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>attribution7</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>attribution8</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>communication1</b></p></td>
+ <td> </td>
+ <td>Fail</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>communication2</b></p></td>
+ <td> </td>
+ <td>Fail</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>communication3</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>communication4</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>communication5</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>communication6</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>communication7</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>delegation1</b></p></td>
+ <td> </td>
+ <td>Fail</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>delegation2</b></p></td>
+ <td> </td>
+ <td>Fail</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>delegation3</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>delegation4</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>delegation5</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>delegation6</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>delegation7</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>delegation8</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>derivation10</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>derivation11</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>derivation12</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>derivation13</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>derivation1</b></p></td>
+ <td> </td>
+ <td>Fail</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>derivation2</b></p></td>
+ <td> </td>
+ <td>Fail</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>derivation3</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>derivation4</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>derivation5</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>derivation6</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>derivation7</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>derivation8</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>derivation9</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>end10</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>end1</b></p></td>
+ <td> </td>
+ <td>Fail</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>end2</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>end3</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>end4</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>end5</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>end6</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>end7</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>end8</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>end9</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>entity10</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>entity1</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>entity2</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>entity3</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>entity4</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>entity5</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>entity6</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>entity7</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>entity8</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>entity9</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>generation1</b></p></td>
+ <td> </td>
+ <td>Fail</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>generation2</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>generation3</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>generation4</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>generation5</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>generation6</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>generation7</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>influence1</b></p></td>
+ <td> </td>
+ <td>Fail</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>influence2</b></p></td>
+ <td> </td>
+ <td>Fail</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>influence3</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>influence4</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>influence5</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>influence6</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>influence7</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>invalidation1</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>invalidation2</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>invalidation3</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>invalidation4</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>invalidation5</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>invalidation6</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>invalidation7</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>member1</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>member2</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>member3</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>mention1</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>mention2</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>specialization1</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>start10</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>start1</b></p></td>
+ <td> </td>
+ <td>Fail</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>start2</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>start3</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>start4</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>start5</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>start6</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>start7</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>start8</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>start9</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>usage1</b></p></td>
+ <td> </td>
+ <td>Fail</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>usage2</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>usage3</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>usage4</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>usage5</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>usage6</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+ <tr>
+ <td><b>usage7</b></p></td>
+ <td> </td>
+ <td>Success</p></td>
+ <td> </td>
+ </tr>
+</table>
+<p>For each report, the successful and failed
+ validations will need to be checked against the expected validation results in Table 2 above . A
+ test case is considered to be successful if its validation result is the same
+as the expected result in Table 2.</p>
+<p>The test case results will also need to be
+ collated against the constraints they cover to indicate which constraints
+ have/have not been successfully tested.</p>
+<p>The collated result will be made available
+ to the implementer soon after they submit their result</p>
+
+</body>
+</html>