JSON-LD Test Suite

Tests are defined into compact, expand, flatten, frame, normalize, and rdf sections:

• compact tests have input, expected and context documents, and may have options. The expected results can be compared using JSON object comparison with the processor output.
• error tests have an input document, and may have a context, frame and options. The expected result is an exception matching an error code defined for JsonLdErrorCode.
• expand tests have input and expected documents, and may have options. The expected results can be compared using JSON object comparison with the processor output.
• flatten tests have input and expected documents, and may have options. The expected results can be compared using JSON object comparison with the processor output.
• frame tests have input, frame and expected documents, and may have options. The expected results can be compared using JSON object comparison with the processor output.
• normalize tests have input and expected documents, and may have options. The expected results can be compared using string comparison with the processor output.
• fromRdf tests have input and expected documents, and may have options. The expected results can be compared using JSON object comparison with the processor output.
• remote-doc tests have input and expected documents, and may have options. These tests are setup somewhat differently, as they depend on specific HTTP header results to invoke document loading behavior. The expected results are either an expected error or result document, depending on if the test type include NegativeEvaluationTest or PositiveEvaluationTest. For negative tests, the expected result is an exception matching an error code defined for JsonLdErrorCode. For positive tests, the expected results can be compared using JSON object comparison with the processor output.
• toRdf tests have input and expected documents, and may have options. The expected results can be compared using string comparison with the processor output.

To run the tests, create a test runner which will run through each test manifest and execute the tests defined within the manifest using the rules associated with each @type defined for the test case as defined in the test vocabulary.

Note that property values are typed, and those which are typed as @id must be treated as IRIs relative to the manifest test base. In particular, this means that input, context, frame, and expandContext are to be passed as IRIs to the API methods being tested.

More details for running different types of tests are also defined in the test vocabulary.

If you would like to contribute a new test or a fix to an existing test, please follow these steps:

1. Notify the JSON-LD mailing list, public-linked-json@w3.org, that you will be creating a new test or fix and the purpose of the change.
2. Clone the git repository: git://github.com/json-ld/json-ld.org.git
3. Make your changes and submit them via GitHub, or via a ‘git format-patch’ to the JSON-LD mailing list.

Optionally, you can ask for direct access to the repository and may make changes directly to the JSON-LD Test Suite source code. All updates to the test suite go live within seconds of committing the changes to GitHub via a WebHook call.

In order to add a unit test, you must follow these steps:

1. Pick a new unit test number. For example - 250. To be consistent, please use the next available unit test number.
2. Create a the input file in the tests/ directory with a .jsonld extension. For example: tests/expand-0250-in.jsonld
3. Create a file containing the expected result in the tests/ directory with a .jsonld or .nq extension. For example: tests/expand-0250-out.jsonld
4. Add your test to the corresponding xxx-manifest.jsonld.

The test suite is designed to empower JSON-LD processor maintainers to create and add tests as they see fit. This may mean that the test suite may become unstable from time to time, but this approach has been taken so that the long-term goal of having a comprehensive test suite for JSON-LD can be achieved by the JSON-LD community.