POC for basic output format protocol & Annotations

[VIDIT45AGARWAL]

Following up on our recent discussions, I’ve put together a POC for implementing the standard basic output format protocol on the Bowtie/Python side. If a test suite has annotations, Bowtie will ask the implementation to return the basic output format instead of just the usual boolean flag.

What's included in this POC:

• Protocol Update (run.json): Added an output field to the Run command to negotiate format ("flag" vs "basic"), and updated the response schema to accept a details array from the harness.
• Polymorphic Test Results (_commands.py): Refactored TestResult into a factory returning either FlagTestResult or BasicTestResult. The new matches() method natively extracts annotations from the details array and compares them to assertions, removing the need for manual collection logic.
• Auto-Detection (_suite.py & _cli.py): Bowtie now detects if a test suite requires annotations. If it does, the CLI automatically appends output: "basic" to the Run requests.

I haven't updated the Python test suite yet. Because I changed TestResult, several of the existing mock tests in bowtie/tests/ are currently failing. Before updating any tests I wanted to get you eyes on it, does this approach looks good??

[read-the-docs-community]

Documentation build overview

📚 Bowtie | 🛠️ Build #32953902 | 📁 Comparing aa252eb against latest (5d95aa1)

  🔍 Preview build  

3 files changed

± cli/index.html
± genindex/index.html
± implementers/index.html

[VIDIT45AGARWAL]

I have now updated the tests also, now it is passing all the checks.

[jdesrosiers]

I don't think coupling annotation support to the basic output format is a good idea. Not all implementations that support annotations support the basic output format. For example, @hyperjump/json-schema supports basic output, but doesn't include annotations in the basic output. Instead there's an API to get the annotations for a given location. Another example is JSONEverything that doesn't implement the basic output format. Instead it implements the a new output format that's intended to replace the 2019/2020 formats with the next release.

[AgniveshChaubey]

Sorry for the delay... I’ve left a few comments. This looks like a good starting point overall. One thing that might also be helpful is updating one of the test harnesses, just to verify that things are working as expected.

[AgniveshChaubey]

Depending on the test suite, a test will either have a valid or assertions property. So maybe we can structure the schema that way? Possibly, we can utilize the oneOf keyword here.

[AgniveshChaubey]

since we know the structure of assertions, instead of leaving it open, maybe we should tighten the constraint here?

[AgniveshChaubey]

Here also, maybe we should standardize the response structure. Otherwise, if implementation responds with a different structure, bowtie might find it difficult to process. However, keeping Jason's point mentioned above in mind, if different implementations include annotations in different output formats, we might still face difficulties processing them.

So, as per Julian's suggestion, we can implement our own spec that harnesses must follow when responding with annotation results. And if we implement our own spec, we can update the output enum to something like ["flag", "rich"].

[AgniveshChaubey]

to decide the output format, I think we can simply check whether the dialect URL refers to the annotation test suite or the validation test suite, instead of determining the output format by digging into one of the test cases.

[AgniveshChaubey]

Another example is JSONEverything that doesn't implement the basic output format. Instead it implements the a new output format that's intended to replace the 2019/2020 formats with the next release.

@jdesrosiers, does that mean even if the schema uses 2020-12 dialect, its corresponding output will follow the next release format? If so, does that mean that the JSON Schemas spec doesn't enforce implementations to produce dialect-specific output, , and that they can use whatever output format they want?

[VIDIT45AGARWAL]

Not all implementations that support annotations support the basic output format

I understand that not all implementations that support annotations natively output the basic format(including annotations) by default.

The primary reason for targeting the basic output format in this POC was to establish a pragmatic, uniform standard that Bowtie can reliably expect from all harnesses.
If we decouple from a standardized format like basic, we run into a significant maintenance hurdle: we would need to update the harnesses independently, understanding and utilizing the annotation-extraction logic for each of the implementations that support annotations. Furthermore, handling annotations via custom harness logic would likely require us to introduce custom flags into the Bowtie protocol (e.g., an "annotations": true flag) - which Julian has explicitly mentioned he wants to avoid.

One alternative would be defining a separate Bowtie spec (Julian's suggestion on slack) that every harness must manually conform to, I am not sure whether doing this is right or not, but this might help achieve the goal of a uniform automated way for Bowtie to receive annotations without having to manage numerous bespoke API wrappers.

If you have any suggestions regarding this, then it would be really helpful.

[jdesrosiers]

does that mean that the JSON Schemas spec doesn't enforce implementations to produce dialect-specific output, , and that they can use whatever output format they want?

That's correct. The output format has always been optional and has never been intended to be draft specific. Implementations are expected to use a consistent output format across all the versions they support. The output format is for third parties to be able to process for various purposes (including but not limited to collecting annotation data). If a schema references another schema that uses a different draft, we wouldn't want it to change how it collects output part way through. The output should be consistent so a third-party output processor can easily process it.

The primary reason for targeting the basic output format in this POC was to establish a pragmatic, uniform standard that Bowtie can reliably expect from all harnesses.

That's going to be far from reliable. Unfortunately, the majority of implementations don't support any of the specified output formats.

If we decouple from a standardized format like basic, we run into a significant maintenance hurdle: we would need to update the harnesses independently, understanding and utilizing the annotation-extraction logic for each of the implementations that support annotations.

My expectation was that each implementation would have to provide its own harness for returning annotations. Bowtie would ask the harness for the annotations for a given keyword at a given location. The harness would return and array of values and compare against the test assertion.

There's no way you're going to be able to just call the validation function and reliably get annotations in a consistent format.

[VIDIT45AGARWAL]

Hey @AgniveshChaubey @Julian I have some updates.

• I have updated the schemas to tighten the assertions by utilizing oneOf
• The Bowtie suite is now configured so that it can pull and run tests directly from the official JSON-Schema-Test-Suite/annotations/tests repository URL.
• Added a validation function that checks whether a specific dialect is compatible for a given test case before running it.
• Updated one of the existing implementations (networknt) to fully support the new annotation protocol.

When I run the following command to test the implementation against Draft 2020-12:

.nox/tests-3-14/bin/bowtie suite \
  -i java-networknt-json-schema-validator \
  --dialect 2020 \
  https://github.com/json-schema-org/JSON-Schema-Test-Suite/tree/main/annotations/tests/

We get the following output:

{"implementations": {"java-networknt-json-schema-validator": {"name": "com.networknt-json-schema-validator", "language": "java", "homepage": "https://github.com/networknt/json-schema-validator/", "issues": "https://github.com/networknt/json-schema-validator/issues", "source": "https://github.com/networknt/json-schema-validator/", "dialects": ["https://json-schema.org/draft/2020-12/schema", "https://json-schema.org/draft/2019-09/schema", "http://json-schema.org/draft-07/schema#", "http://json-schema.org/draft-06/schema#", "http://json-schema.org/draft-04/schema#"], "version": "3.0.2", "language_version": "25.0.3+11-LTS", "os": "Linux", "os_version": "7.0.0-15-generic", "documentation": "https://doc.networknt.com/library/json-schema-validator/", "links": []}}, "bowtie_version": "0.1.dev8024+g1dd6e7635", "metadata": {}, "dialect": "https://json-schema.org/draft/2020-12/schema", "started": "2026-05-29T06:13:22.436572+00:00"}
{"seq": 1, "case": {"description": "`properties`, `patternProperties`, and `additionalProperties`", "schema": {"properties": {"foo": {"title": "Foo"}}, "patternProperties": {"^a": {"title": "Bar"}}, "additionalProperties": {"title": "Baz"}}, "tests": [{"description": "", "instance": {}, "assertions": [{"location": "/foo", "keyword": "title", "expected": {}}, {"location": "/apple", "keyword": "title", "expected": {}}, {"location": "/bar", "keyword": "title", "expected": {}}]}, {"description": "", "instance": {"foo": {}, "apple": {}, "baz": {}}, "assertions": [{"location": "/foo", "keyword": "title", "expected": {"#/properties/foo": "Foo"}}, {"location": "/apple", "keyword": "title", "expected": {"#/patternProperties/%5Ea": "Bar"}}, {"location": "/baz", "keyword": "title", "expected": {"#/additionalProperties": "Baz"}}]}]}}
{"seq": 2, "case": {"description": "`propertyNames` doesn't annotate property values", "schema": {"propertyNames": {"const": "foo", "title": "Foo"}}, "tests": [{"description": "", "instance": {"foo": 42}, "assertions": [{"location": "/foo", "keyword": "title", "expected": {}}]}]}}
{"seq": 3, "case": {"description": "`prefixItems` and `items`", "schema": {"prefixItems": [{"title": "Foo"}], "items": {"title": "Bar"}}, "tests": [{"description": "", "instance": ["foo", "bar"], "assertions": [{"location": "/0", "keyword": "title", "expected": {"#/prefixItems/0": "Foo"}}, {"location": "/1", "keyword": "title", "expected": {"#/items": "Bar"}}, {"location": "/2", "keyword": "title", "expected": {}}]}]}}
{"seq": 4, "case": {"description": "`contains`", "schema": {"contains": {"type": "number", "title": "Foo"}}, "tests": [{"description": "", "instance": ["foo", 42, true], "assertions": [{"location": "/0", "keyword": "title", "expected": {}}, {"location": "/1", "keyword": "title", "expected": {"#/contains": "Foo"}}, {"location": "/2", "keyword": "title", "expected": {}}, {"location": "/3", "keyword": "title", "expected": {}}]}]}}
{"seq": 5, "case": {"description": "`allOf`", "schema": {"allOf": [{"title": "Foo"}, {"title": "Bar"}]}, "tests": [{"description": "", "instance": "foo", "assertions": [{"location": "", "keyword": "title", "expected": {"#/allOf/1": "Bar", "#/allOf/0": "Foo"}}]}]}}
{"seq": 6, "case": {"description": "`anyOf`", "schema": {"anyOf": [{"type": "integer", "title": "Foo"}, {"type": "number", "title": "Bar"}]}, "tests": [{"description": "", "instance": 42, "assertions": [{"location": "", "keyword": "title", "expected": {"#/anyOf/1": "Bar", "#/anyOf/0": "Foo"}}]}, {"description": "", "instance": 4.2, "assertions": [{"location": "", "keyword": "title", "expected": {"#/anyOf/1": "Bar"}}]}]}}
{"seq": 7, "case": {"description": "`oneOf`", "schema": {"oneOf": [{"type": "string", "title": "Foo"}, {"type": "number", "title": "Bar"}]}, "tests": [{"description": "", "instance": "foo", "assertions": [{"location": "", "keyword": "title", "expected": {"#/oneOf/0": "Foo"}}]}, {"description": "", "instance": 42, "assertions": [{"location": "", "keyword": "title", "expected": {"#/oneOf/1": "Bar"}}]}]}}
{"seq": 8, "case": {"description": "`not`", "schema": {"title": "Foo", "not": {"not": {"title": "Bar"}}}, "tests": [{"description": "", "instance": {}, "assertions": [{"location": "", "keyword": "title", "expected": {"#": "Foo"}}]}]}}
{"seq": 9, "case": {"description": "`dependentSchemas`", "schema": {"dependentSchemas": {"foo": {"title": "Foo"}}}, "tests": [{"description": "", "instance": {"foo": 42}, "assertions": [{"location": "", "keyword": "title", "expected": {"#/dependentSchemas/foo": "Foo"}}]}, {"description": "", "instance": {"foo": 42}, "assertions": [{"location": "/foo", "keyword": "title", "expected": {}}]}]}}
{"seq": 10, "case": {"description": "`if`, `then`, and `else`", "schema": {"if": {"title": "If", "type": "string"}, "then": {"title": "Then"}, "else": {"title": "Else"}}, "tests": [{"description": "", "instance": "foo", "assertions": [{"location": "", "keyword": "title", "expected": {"#/then": "Then", "#/if": "If"}}]}, {"description": "", "instance": 42, "assertions": [{"location": "", "keyword": "title", "expected": {"#/else": "Else"}}]}]}}
{"seq": 11, "case": {"description": "`contentMediaType` is an annotation for string instances", "schema": {"contentMediaType": "application/json"}, "tests": [{"description": "", "instance": "{ \"foo\": \"bar\" }", "assertions": [{"location": "", "keyword": "contentMediaType", "expected": {"#": "application/json"}}]}, {"description": "", "instance": 42, "assertions": [{"location": "", "keyword": "contentMediaType", "expected": {}}]}]}}
{"seq": 12, "case": {"description": "`contentEncoding` is an annotation for string instances", "schema": {"contentEncoding": "base64"}, "tests": [{"description": "", "instance": "SGVsbG8gZnJvbSBKU09OIFNjaGVtYQ==", "assertions": [{"location": "", "keyword": "contentEncoding", "expected": {"#": "base64"}}]}, {"description": "", "instance": 42, "assertions": [{"location": "", "keyword": "contentEncoding", "expected": {}}]}]}}
{"seq": 13, "case": {"description": "`contentSchema` is an annotation for string instances", "schema": {"contentMediaType": "application/json", "contentSchema": {"type": "number"}}, "tests": [{"description": "", "instance": "42", "assertions": [{"location": "", "keyword": "contentSchema", "expected": {"#": {"type": "number"}}}]}, {"description": "", "instance": 42, "assertions": [{"location": "", "keyword": "contentSchema", "expected": {}}]}]}}
{"seq": 14, "case": {"description": "`contentSchema` requires `contentMediaType`", "schema": {"contentSchema": {"type": "number"}}, "tests": [{"description": "", "instance": "42", "assertions": [{"location": "", "keyword": "contentSchema", "expected": {}}]}]}}
{"seq": 15, "case": {"description": "`$ref` and `$defs`", "schema": {"$ref": "#/$defs/foo", "$defs": {"foo": {"title": "Foo"}}}, "tests": [{"description": "", "instance": "foo", "assertions": [{"location": "", "keyword": "title", "expected": {"#/$defs/foo": "Foo"}}]}]}}
{"seq": 16, "case": {"description": "`$dynamicRef` resolves to `$dynamicAnchor`", "schema": {"$dynamicRef": "#foo", "$defs": {"foo": {"$dynamicAnchor": "foo", "title": "Foo"}}}, "tests": [{"description": "", "instance": "bar", "assertions": [{"location": "", "keyword": "title", "expected": {"#/$defs/foo": "Foo"}}]}]}}
{"seq": 17, "case": {"description": "`$dynamicRef` resolves to different `$dynamicAnchor`s depending on dynamic path", "schema": {"$id": "https://test.json-schema.org/dynamic-ref-annotation/main", "if": {"properties": {"kindOfList": {"const": "numbers"}}, "required": ["kindOfList"]}, "then": {"$ref": "numberList"}, "else": {"$ref": "stringList"}, "$defs": {"genericList": {"$id": "genericList", "properties": {"list": {"items": {"$dynamicRef": "#itemType"}}}, "$defs": {"defaultItemType": {"$dynamicAnchor": "itemType"}}}, "numberList": {"$id": "numberList", "$defs": {"itemType": {"$dynamicAnchor": "itemType", "title": "Number Item"}}, "$ref": "genericList"}, "stringList": {"$id": "stringList", "$defs": {"itemType": {"$dynamicAnchor": "itemType", "title": "String Item"}}, "$ref": "genericList"}}}, "tests": [{"description": "", "instance": {"kindOfList": "numbers", "list": [1]}, "assertions": [{"location": "/list/0", "keyword": "title", "expected": {"#/$defs/numberList/$defs/itemType": "Number Item"}}]}, {"description": "", "instance": {"kindOfList": "strings", "list": ["foo"]}, "assertions": [{"location": "/list/0", "keyword": "title", "expected": {"#/$defs/stringList/$defs/itemType": "String Item"}}]}]}}
{"seq": 18, "case": {"description": "`format` is an annotation", "schema": {"format": "email"}, "tests": [{"description": "", "instance": "foo@bar.com", "assertions": [{"location": "", "keyword": "format", "expected": {"#": "email"}}]}]}}
{"seq": 19, "case": {"description": "`title` is an annotation", "schema": {"title": "Foo"}, "tests": [{"description": "", "instance": 42, "assertions": [{"location": "", "keyword": "title", "expected": {"#": "Foo"}}]}]}}
{"seq": 20, "case": {"description": "`description` is an annotation", "schema": {"description": "Foo"}, "tests": [{"description": "", "instance": 42, "assertions": [{"location": "", "keyword": "description", "expected": {"#": "Foo"}}]}]}}
{"seq": 21, "case": {"description": "`default` is an annotation", "schema": {"default": "Foo"}, "tests": [{"description": "", "instance": 42, "assertions": [{"location": "", "keyword": "default", "expected": {"#": "Foo"}}]}]}}
{"seq": 22, "case": {"description": "`deprecated` is an annotation", "schema": {"deprecated": true}, "tests": [{"description": "", "instance": 42, "assertions": [{"location": "", "keyword": "deprecated", "expected": {"#": true}}]}]}}
{"seq": 23, "case": {"description": "`readOnly` is an annotation", "schema": {"readOnly": true}, "tests": [{"description": "", "instance": 42, "assertions": [{"location": "", "keyword": "readOnly", "expected": {"#": true}}]}]}}
{"seq": 24, "case": {"description": "`writeOnly` is an annotation", "schema": {"writeOnly": true}, "tests": [{"description": "", "instance": 42, "assertions": [{"location": "", "keyword": "writeOnly", "expected": {"#": true}}]}]}}
{"seq": 25, "case": {"description": "`examples` is an annotation", "schema": {"examples": ["Foo", "Bar"]}, "tests": [{"description": "", "instance": "Foo", "assertions": [{"location": "", "keyword": "examples", "expected": {"#": ["Foo", "Bar"]}}]}]}}
{"seq": 26, "case": {"description": "`unevaluatedProperties` alone", "schema": {"unevaluatedProperties": {"title": "Unevaluated"}}, "tests": [{"description": "", "instance": {"foo": 42, "bar": 24}, "assertions": [{"location": "/foo", "keyword": "title", "expected": {"#/unevaluatedProperties": "Unevaluated"}}, {"location": "/bar", "keyword": "title", "expected": {"#/unevaluatedProperties": "Unevaluated"}}]}]}}
{"seq": 27, "case": {"description": "`unevaluatedProperties` with `properties`", "schema": {"properties": {"foo": {"title": "Evaluated"}}, "unevaluatedProperties": {"title": "Unevaluated"}}, "tests": [{"description": "", "instance": {"foo": 42, "bar": 24}, "assertions": [{"location": "/foo", "keyword": "title", "expected": {"#/properties/foo": "Evaluated"}}, {"location": "/bar", "keyword": "title", "expected": {"#/unevaluatedProperties": "Unevaluated"}}]}]}}
{"seq": 28, "case": {"description": "`unevaluatedProperties` with `patternProperties`", "schema": {"patternProperties": {"^a": {"title": "Evaluated"}}, "unevaluatedProperties": {"title": "Unevaluated"}}, "tests": [{"description": "", "instance": {"apple": 42, "bar": 24}, "assertions": [{"location": "/apple", "keyword": "title", "expected": {"#/patternProperties/%5Ea": "Evaluated"}}, {"location": "/bar", "keyword": "title", "expected": {"#/unevaluatedProperties": "Unevaluated"}}]}]}}
{"seq": 29, "case": {"description": "`unevaluatedProperties` with `additionalProperties`", "schema": {"additionalProperties": {"title": "Evaluated"}, "unevaluatedProperties": {"title": "Unevaluated"}}, "tests": [{"description": "", "instance": {"foo": 42, "bar": 24}, "assertions": [{"location": "/foo", "keyword": "title", "expected": {"#/additionalProperties": "Evaluated"}}, {"location": "/bar", "keyword": "title", "expected": {"#/additionalProperties": "Evaluated"}}]}]}}
{"seq": 30, "case": {"description": "`unevaluatedProperties` with `dependentSchemas`", "schema": {"dependentSchemas": {"foo": {"properties": {"bar": {"title": "Evaluated"}}}}, "unevaluatedProperties": {"title": "Unevaluated"}}, "tests": [{"description": "", "instance": {"foo": 42, "bar": 24}, "assertions": [{"location": "/foo", "keyword": "title", "expected": {"#/unevaluatedProperties": "Unevaluated"}}, {"location": "/bar", "keyword": "title", "expected": {"#/dependentSchemas/foo/properties/bar": "Evaluated"}}]}]}}
{"seq": 31, "case": {"description": "`unevaluatedProperties` with `if`, `then`, and `else`", "schema": {"if": {"properties": {"foo": {"type": "string", "title": "If"}}}, "then": {"properties": {"foo": {"title": "Then"}}}, "else": {"properties": {"foo": {"title": "Else"}}}, "unevaluatedProperties": {"title": "Unevaluated"}}, "tests": [{"description": "", "instance": {"foo": "", "bar": 42}, "assertions": [{"location": "/foo", "keyword": "title", "expected": {"#/then/properties/foo": "Then", "#/if/properties/foo": "If"}}, {"location": "/bar", "keyword": "title", "expected": {"#/unevaluatedProperties": "Unevaluated"}}]}, {"description": "", "instance": {"foo": 42, "bar": ""}, "assertions": [{"location": "/foo", "keyword": "title", "expected": {"#/else/properties/foo": "Else"}}, {"location": "/bar", "keyword": "title", "expected": {"#/unevaluatedProperties": "Unevaluated"}}]}]}}
{"seq": 32, "case": {"description": "`unevaluatedProperties` with `allOf`", "schema": {"allOf": [{"properties": {"foo": {"title": "Evaluated"}}}], "unevaluatedProperties": {"title": "Unevaluated"}}, "tests": [{"description": "", "instance": {"foo": 42, "bar": 24}, "assertions": [{"location": "/foo", "keyword": "title", "expected": {"#/allOf/0/properties/foo": "Evaluated"}}, {"location": "/bar", "keyword": "title", "expected": {"#/unevaluatedProperties": "Unevaluated"}}]}]}}
{"seq": 33, "case": {"description": "`unevaluatedProperties` with `anyOf`", "schema": {"anyOf": [{"properties": {"foo": {"title": "Evaluated"}}}], "unevaluatedProperties": {"title": "Unevaluated"}}, "tests": [{"description": "", "instance": {"foo": 42, "bar": 24}, "assertions": [{"location": "/foo", "keyword": "title", "expected": {"#/anyOf/0/properties/foo": "Evaluated"}}, {"location": "/bar", "keyword": "title", "expected": {"#/unevaluatedProperties": "Unevaluated"}}]}]}}
{"seq": 34, "case": {"description": "`unevaluatedProperties` with `oneOf`", "schema": {"oneOf": [{"properties": {"foo": {"title": "Evaluated"}}}], "unevaluatedProperties": {"title": "Unevaluated"}}, "tests": [{"description": "", "instance": {"foo": 42, "bar": 24}, "assertions": [{"location": "/foo", "keyword": "title", "expected": {"#/oneOf/0/properties/foo": "Evaluated"}}, {"location": "/bar", "keyword": "title", "expected": {"#/unevaluatedProperties": "Unevaluated"}}]}]}}
{"seq": 35, "case": {"description": "`unevaluatedProperties` with `not`", "schema": {"not": {"not": {"properties": {"foo": {"title": "Evaluated"}}}}, "unevaluatedProperties": {"title": "Unevaluated"}}, "tests": [{"description": "", "instance": {"foo": 42, "bar": 24}, "assertions": [{"location": "/foo", "keyword": "title", "expected": {"#/unevaluatedProperties": "Unevaluated"}}, {"location": "/bar", "keyword": "title", "expected": {"#/unevaluatedProperties": "Unevaluated"}}]}]}}
{"seq": 36, "case": {"description": "`unevaluatedItems` alone", "schema": {"unevaluatedItems": {"title": "Unevaluated"}}, "tests": [{"description": "", "instance": [42, 24], "assertions": [{"location": "/0", "keyword": "title", "expected": {"#/unevaluatedItems": "Unevaluated"}}, {"location": "/1", "keyword": "title", "expected": {"#/unevaluatedItems": "Unevaluated"}}]}]}}
{"seq": 37, "case": {"description": "`unevaluatedItems` with `prefixItems`", "schema": {"prefixItems": [{"title": "Evaluated"}], "unevaluatedItems": {"title": "Unevaluated"}}, "tests": [{"description": "", "instance": [42, 24], "assertions": [{"location": "/0", "keyword": "title", "expected": {"#/prefixItems/0": "Evaluated"}}, {"location": "/1", "keyword": "title", "expected": {"#/unevaluatedItems": "Unevaluated"}}]}]}}
{"seq": 38, "case": {"description": "`unevaluatedItems` with `contains`", "schema": {"contains": {"type": "string", "title": "Evaluated"}, "unevaluatedItems": {"title": "Unevaluated"}}, "tests": [{"description": "", "instance": ["foo", 42], "assertions": [{"location": "/0", "keyword": "title", "expected": {"#/contains": "Evaluated"}}, {"location": "/1", "keyword": "title", "expected": {"#/unevaluatedItems": "Unevaluated"}}]}]}}
{"seq": 39, "case": {"description": "`unevaluatedItems` with `if`, `then`, and `else`", "schema": {"if": {"prefixItems": [{"type": "string", "title": "If"}]}, "then": {"prefixItems": [{"title": "Then"}]}, "else": {"prefixItems": [{"title": "Else"}]}, "unevaluatedItems": {"title": "Unevaluated"}}, "tests": [{"description": "", "instance": ["", 42], "assertions": [{"location": "/0", "keyword": "title", "expected": {"#/then/prefixItems/0": "Then", "#/if/prefixItems/0": "If"}}, {"location": "/1", "keyword": "title", "expected": {"#/unevaluatedItems": "Unevaluated"}}]}, {"description": "", "instance": [42, ""], "assertions": [{"location": "/0", "keyword": "title", "expected": {"#/else/prefixItems/0": "Else"}}, {"location": "/1", "keyword": "title", "expected": {"#/unevaluatedItems": "Unevaluated"}}]}]}}
{"seq": 40, "case": {"description": "`unevaluatedItems` with `allOf`", "schema": {"allOf": [{"prefixItems": [{"title": "Evaluated"}]}], "unevaluatedItems": {"title": "Unevaluated"}}, "tests": [{"description": "", "instance": [42, 24], "assertions": [{"location": "/0", "keyword": "title", "expected": {"#/allOf/0/prefixItems/0": "Evaluated"}}, {"location": "/1", "keyword": "title", "expected": {"#/unevaluatedItems": "Unevaluated"}}]}]}}
{"seq": 41, "case": {"description": "`unevaluatedItems` with `anyOf`", "schema": {"anyOf": [{"prefixItems": [{"title": "Evaluated"}]}], "unevaluatedItems": {"title": "Unevaluated"}}, "tests": [{"description": "", "instance": [42, 24], "assertions": [{"location": "/0", "keyword": "title", "expected": {"#/anyOf/0/prefixItems/0": "Evaluated"}}, {"location": "/1", "keyword": "title", "expected": {"#/unevaluatedItems": "Unevaluated"}}]}]}}
{"seq": 42, "case": {"description": "`unevaluatedItems` with `oneOf`", "schema": {"oneOf": [{"prefixItems": [{"title": "Evaluated"}]}], "unevaluatedItems": {"title": "Unevaluated"}}, "tests": [{"description": "", "instance": [42, 24], "assertions": [{"location": "/0", "keyword": "title", "expected": {"#/oneOf/0/prefixItems/0": "Evaluated"}}, {"location": "/1", "keyword": "title", "expected": {"#/unevaluatedItems": "Unevaluated"}}]}]}}
{"seq": 43, "case": {"description": "`unevaluatedItems` with `not`", "schema": {"not": {"not": {"prefixItems": [{"title": "Evaluated"}]}}, "unevaluatedItems": {"title": "Unevaluated"}}, "tests": [{"description": "", "instance": [42, 24], "assertions": [{"location": "/0", "keyword": "title", "expected": {"#/unevaluatedItems": "Unevaluated"}}, {"location": "/1", "keyword": "title", "expected": {"#/unevaluatedItems": "Unevaluated"}}]}]}}
{"seq": 44, "case": {"description": "`unknownKeyword` is an annotation", "schema": {"$schema": "https://json-schema.org/draft/2020-12/schema", "x-unknownKeyword": "Foo"}, "tests": [{"description": "", "instance": 42, "assertions": [{"location": "", "keyword": "x-unknownKeyword", "expected": {"#": "Foo"}}]}]}}
{"seq": 1, "implementation": "java-networknt-json-schema-validator", "expected": [[{"location": "/foo", "keyword": "title", "expected": {}}, {"location": "/apple", "keyword": "title", "expected": {}}, {"location": "/bar", "keyword": "title", "expected": {}}], [{"location": "/foo", "keyword": "title", "expected": {"#/properties/foo": "Foo"}}, {"location": "/apple", "keyword": "title", "expected": {"#/patternProperties/%5Ea": "Bar"}}, {"location": "/baz", "keyword": "title", "expected": {"#/additionalProperties": "Baz"}}]], "results": [{"valid": true, "annotations": [{"annotation": [], "keywordLocation": "#/properties", "instanceLocation": "", "keyword": "properties"}, {"annotation": [], "keywordLocation": "#/patternProperties", "instanceLocation": "", "keyword": "patternProperties"}, {"annotation": [], "keywordLocation": "#/additionalProperties", "instanceLocation": "", "keyword": "additionalProperties"}]}, {"valid": true, "annotations": [{"annotation": "Foo", "keywordLocation": "#/properties/foo/title", "instanceLocation": "/foo", "keyword": "title"}, {"annotation": ["foo"], "keywordLocation": "#/properties", "instanceLocation": "", "keyword": "properties"}, {"annotation": ["apple"], "keywordLocation": "#/patternProperties", "instanceLocation": "", "keyword": "patternProperties"}, {"annotation": ["foo", "apple", "baz"], "keywordLocation": "#/additionalProperties", "instanceLocation": "", "keyword": "additionalProperties"}, {"annotation": "Bar", "keywordLocation": "#/patternProperties/^a/title", "instanceLocation": "/apple", "keyword": "title"}, {"annotation": "Baz", "keywordLocation": "#/additionalProperties/title", "instanceLocation": "/baz", "keyword": "title"}]}]}
{"seq": 2, "implementation": "java-networknt-json-schema-validator", "expected": [[{"location": "/foo", "keyword": "title", "expected": {}}]], "results": [{"valid": true, "annotations": [{"annotation": "Foo", "keywordLocation": "#/propertyNames/title", "instanceLocation": "/foo", "keyword": "title"}]}]}
{"seq": 3, "implementation": "java-networknt-json-schema-validator", "expected": [[{"location": "/0", "keyword": "title", "expected": {"#/prefixItems/0": "Foo"}}, {"location": "/1", "keyword": "title", "expected": {"#/items": "Bar"}}, {"location": "/2", "keyword": "title", "expected": {}}]], "results": [{"valid": true, "annotations": [{"annotation": "Foo", "keywordLocation": "#/prefixItems/0/title", "instanceLocation": "/0", "keyword": "title"}, {"annotation": 1, "keywordLocation": "#/prefixItems", "instanceLocation": "", "keyword": "prefixItems"}, {"annotation": true, "keywordLocation": "#/items", "instanceLocation": "", "keyword": "items"}, {"annotation": "Bar", "keywordLocation": "#/items/title", "instanceLocation": "/1", "keyword": "title"}]}]}
{"seq": 4, "implementation": "java-networknt-json-schema-validator", "expected": [[{"location": "/0", "keyword": "title", "expected": {}}, {"location": "/1", "keyword": "title", "expected": {"#/contains": "Foo"}}, {"location": "/2", "keyword": "title", "expected": {}}, {"location": "/3", "keyword": "title", "expected": {}}]], "results": [{"valid": true, "annotations": [{"annotation": "Foo", "keywordLocation": "#/contains/title", "instanceLocation": "/1", "keyword": "title"}, {"annotation": [1], "keywordLocation": "#/contains", "instanceLocation": "", "keyword": "contains"}]}]}
{"seq": 5, "implementation": "java-networknt-json-schema-validator", "expected": [[{"location": "", "keyword": "title", "expected": {"#/allOf/1": "Bar", "#/allOf/0": "Foo"}}]], "results": [{"valid": true, "annotations": [{"annotation": "Foo", "keywordLocation": "#/allOf/0/title", "instanceLocation": "", "keyword": "title"}, {"annotation": "Bar", "keywordLocation": "#/allOf/1/title", "instanceLocation": "", "keyword": "title"}]}]}
{"seq": 6, "implementation": "java-networknt-json-schema-validator", "expected": [[{"location": "", "keyword": "title", "expected": {"#/anyOf/1": "Bar", "#/anyOf/0": "Foo"}}], [{"location": "", "keyword": "title", "expected": {"#/anyOf/1": "Bar"}}]], "results": [{"valid": true, "annotations": [{"annotation": "Foo", "keywordLocation": "#/anyOf/0/title", "instanceLocation": "", "keyword": "title"}, {"annotation": "Bar", "keywordLocation": "#/anyOf/1/title", "instanceLocation": "", "keyword": "title"}]}, {"valid": true, "annotations": [{"annotation": "Bar", "keywordLocation": "#/anyOf/1/title", "instanceLocation": "", "keyword": "title"}]}]}
{"seq": 7, "implementation": "java-networknt-json-schema-validator", "expected": [[{"location": "", "keyword": "title", "expected": {"#/oneOf/0": "Foo"}}], [{"location": "", "keyword": "title", "expected": {"#/oneOf/1": "Bar"}}]], "results": [{"valid": true, "annotations": [{"annotation": "Foo", "keywordLocation": "#/oneOf/0/title", "instanceLocation": "", "keyword": "title"}]}, {"valid": true, "annotations": [{"annotation": "Bar", "keywordLocation": "#/oneOf/1/title", "instanceLocation": "", "keyword": "title"}]}]}
{"seq": 8, "implementation": "java-networknt-json-schema-validator", "expected": [[{"location": "", "keyword": "title", "expected": {"#": "Foo"}}]], "results": [{"valid": true, "annotations": [{"annotation": "Foo", "keywordLocation": "#/title", "instanceLocation": "", "keyword": "title"}]}]}
{"seq": 9, "implementation": "java-networknt-json-schema-validator", "expected": [[{"location": "", "keyword": "title", "expected": {"#/dependentSchemas/foo": "Foo"}}], [{"location": "/foo", "keyword": "title", "expected": {}}]], "results": [{"valid": true, "annotations": [{"annotation": "Foo", "keywordLocation": "#/dependentSchemas/foo/title", "instanceLocation": "", "keyword": "title"}]}, {"valid": true, "annotations": [{"annotation": "Foo", "keywordLocation": "#/dependentSchemas/foo/title", "instanceLocation": "", "keyword": "title"}]}]}
{"seq": 10, "implementation": "java-networknt-json-schema-validator", "expected": [[{"location": "", "keyword": "title", "expected": {"#/then": "Then", "#/if": "If"}}], [{"location": "", "keyword": "title", "expected": {"#/else": "Else"}}]], "results": [{"valid": true, "annotations": [{"annotation": "If", "keywordLocation": "#/if/title", "instanceLocation": "", "keyword": "title"}, {"annotation": "Then", "keywordLocation": "#/then/title", "instanceLocation": "", "keyword": "title"}]}, {"valid": true, "annotations": [{"annotation": "Else", "keywordLocation": "#/else/title", "instanceLocation": "", "keyword": "title"}]}]}
{"seq": 11, "implementation": "java-networknt-json-schema-validator", "expected": [[{"location": "", "keyword": "contentMediaType", "expected": {"#": "application/json"}}], [{"location": "", "keyword": "contentMediaType", "expected": {}}]], "results": [{"valid": true, "annotations": [{"annotation": "application/json", "keywordLocation": "#/contentMediaType", "instanceLocation": "", "keyword": "contentMediaType"}]}, {"valid": true, "annotations": [{"annotation": "application/json", "keywordLocation": "#/contentMediaType", "instanceLocation": "", "keyword": "contentMediaType"}]}]}
{"seq": 12, "implementation": "java-networknt-json-schema-validator", "expected": [[{"location": "", "keyword": "contentEncoding", "expected": {"#": "base64"}}], [{"location": "", "keyword": "contentEncoding", "expected": {}}]], "results": [{"valid": true, "annotations": [{"annotation": "base64", "keywordLocation": "#/contentEncoding", "instanceLocation": "", "keyword": "contentEncoding"}]}, {"valid": true, "annotations": [{"annotation": "base64", "keywordLocation": "#/contentEncoding", "instanceLocation": "", "keyword": "contentEncoding"}]}]}
{"seq": 13, "implementation": "java-networknt-json-schema-validator", "expected": [[{"location": "", "keyword": "contentSchema", "expected": {"#": {"type": "number"}}}], [{"location": "", "keyword": "contentSchema", "expected": {}}]], "results": [{"valid": true, "annotations": [{"annotation": "application/json", "keywordLocation": "#/contentMediaType", "instanceLocation": "", "keyword": "contentMediaType"}, {"annotation": {"type": "number"}, "keywordLocation": "#/contentSchema", "instanceLocation": "", "keyword": "contentSchema"}]}, {"valid": true, "annotations": [{"annotation": "application/json", "keywordLocation": "#/contentMediaType", "instanceLocation": "", "keyword": "contentMediaType"}, {"annotation": {"type": "number"}, "keywordLocation": "#/contentSchema", "instanceLocation": "", "keyword": "contentSchema"}]}]}
{"seq": 14, "implementation": "java-networknt-json-schema-validator", "expected": [[{"location": "", "keyword": "contentSchema", "expected": {}}]], "results": [{"valid": true, "annotations": [{"annotation": {"type": "number"}, "keywordLocation": "#/contentSchema", "instanceLocation": "", "keyword": "contentSchema"}]}]}
{"seq": 15, "implementation": "java-networknt-json-schema-validator", "expected": [[{"location": "", "keyword": "title", "expected": {"#/$defs/foo": "Foo"}}]], "results": [{"valid": true, "annotations": [{"annotation": "Foo", "keywordLocation": "#/$defs/foo/title", "instanceLocation": "", "keyword": "title"}]}]}
{"seq": 16, "implementation": "java-networknt-json-schema-validator", "expected": [[{"location": "", "keyword": "title", "expected": {"#/$defs/foo": "Foo"}}]], "results": [{"valid": true, "annotations": [{"annotation": "Foo", "keywordLocation": "#/$defs/foo/title", "instanceLocation": "", "keyword": "title"}]}]}
{"seq": 17, "implementation": "java-networknt-json-schema-validator", "expected": [[{"location": "/list/0", "keyword": "title", "expected": {"#/$defs/numberList/$defs/itemType": "Number Item"}}], [{"location": "/list/0", "keyword": "title", "expected": {"#/$defs/stringList/$defs/itemType": "String Item"}}]], "results": [{"valid": true, "annotations": [{"annotation": ["kindOfList"], "keywordLocation": "https://test.json-schema.org/dynamic-ref-annotation/main#/if/properties", "instanceLocation": "", "keyword": "properties"}, {"annotation": ["list"], "keywordLocation": "https://test.json-schema.org/dynamic-ref-annotation/genericList#/properties", "instanceLocation": "", "keyword": "properties"}, {"annotation": "Number Item", "keywordLocation": "https://test.json-schema.org/dynamic-ref-annotation/numberList#/$defs/itemType/title", "instanceLocation": "/list/0", "keyword": "title"}, {"annotation": true, "keywordLocation": "https://test.json-schema.org/dynamic-ref-annotation/genericList#/properties/list/items", "instanceLocation": "/list", "keyword": "items"}]}, {"valid": true, "annotations": [{"annotation": ["list"], "keywordLocation": "https://test.json-schema.org/dynamic-ref-annotation/genericList#/properties", "instanceLocation": "", "keyword": "properties"}, {"annotation": "String Item", "keywordLocation": "https://test.json-schema.org/dynamic-ref-annotation/stringList#/$defs/itemType/title", "instanceLocation": "/list/0", "keyword": "title"}, {"annotation": true, "keywordLocation": "https://test.json-schema.org/dynamic-ref-annotation/genericList#/properties/list/items", "instanceLocation": "/list", "keyword": "items"}]}]}
{"seq": 18, "implementation": "java-networknt-json-schema-validator", "expected": [[{"location": "", "keyword": "format", "expected": {"#": "email"}}]], "results": [{"valid": true, "annotations": [{"annotation": "email", "keywordLocation": "#/format", "instanceLocation": "", "keyword": "format"}]}]}
{"seq": 19, "implementation": "java-networknt-json-schema-validator", "expected": [[{"location": "", "keyword": "title", "expected": {"#": "Foo"}}]], "results": [{"valid": true, "annotations": [{"annotation": "Foo", "keywordLocation": "#/title", "instanceLocation": "", "keyword": "title"}]}]}
{"seq": 20, "implementation": "java-networknt-json-schema-validator", "expected": [[{"location": "", "keyword": "description", "expected": {"#": "Foo"}}]], "results": [{"valid": true, "annotations": [{"annotation": "Foo", "keywordLocation": "#/description", "instanceLocation": "", "keyword": "description"}]}]}
{"seq": 21, "implementation": "java-networknt-json-schema-validator", "expected": [[{"location": "", "keyword": "default", "expected": {"#": "Foo"}}]], "results": [{"valid": true, "annotations": [{"annotation": "Foo", "keywordLocation": "#/default", "instanceLocation": "", "keyword": "default"}]}]}
{"seq": 22, "implementation": "java-networknt-json-schema-validator", "expected": [[{"location": "", "keyword": "deprecated", "expected": {"#": true}}]], "results": [{"valid": true, "annotations": []}]}
{"seq": 23, "implementation": "java-networknt-json-schema-validator", "expected": [[{"location": "", "keyword": "readOnly", "expected": {"#": true}}]], "results": [{"valid": true, "annotations": []}]}
{"seq": 24, "implementation": "java-networknt-json-schema-validator", "expected": [[{"location": "", "keyword": "writeOnly", "expected": {"#": true}}]], "results": [{"valid": true, "annotations": []}]}
{"seq": 25, "implementation": "java-networknt-json-schema-validator", "expected": [[{"location": "", "keyword": "examples", "expected": {"#": ["Foo", "Bar"]}}]], "results": [{"valid": true, "annotations": []}]}
{"seq": 26, "implementation": "java-networknt-json-schema-validator", "expected": [[{"location": "/foo", "keyword": "title", "expected": {"#/unevaluatedProperties": "Unevaluated"}}, {"location": "/bar", "keyword": "title", "expected": {"#/unevaluatedProperties": "Unevaluated"}}]], "results": [{"valid": true, "annotations": [{"annotation": "Unevaluated", "keywordLocation": "#/unevaluatedProperties/title", "instanceLocation": "/foo", "keyword": "title"}, {"annotation": "Unevaluated", "keywordLocation": "#/unevaluatedProperties/title", "instanceLocation": "/bar", "keyword": "title"}, {"annotation": ["foo", "bar"], "keywordLocation": "#/unevaluatedProperties", "instanceLocation": "", "keyword": "unevaluatedProperties"}]}]}
{"seq": 27, "implementation": "java-networknt-json-schema-validator", "expected": [[{"location": "/foo", "keyword": "title", "expected": {"#/properties/foo": "Evaluated"}}, {"location": "/bar", "keyword": "title", "expected": {"#/unevaluatedProperties": "Unevaluated"}}]], "results": [{"valid": true, "annotations": [{"annotation": "Evaluated", "keywordLocation": "#/properties/foo/title", "instanceLocation": "/foo", "keyword": "title"}, {"annotation": ["foo"], "keywordLocation": "#/properties", "instanceLocation": "", "keyword": "properties"}, {"annotation": ["bar"], "keywordLocation": "#/unevaluatedProperties", "instanceLocation": "", "keyword": "unevaluatedProperties"}, {"annotation": "Unevaluated", "keywordLocation": "#/unevaluatedProperties/title", "instanceLocation": "/bar", "keyword": "title"}]}]}
{"seq": 28, "implementation": "java-networknt-json-schema-validator", "expected": [[{"location": "/apple", "keyword": "title", "expected": {"#/patternProperties/%5Ea": "Evaluated"}}, {"location": "/bar", "keyword": "title", "expected": {"#/unevaluatedProperties": "Unevaluated"}}]], "results": [{"valid": true, "annotations": [{"annotation": "Evaluated", "keywordLocation": "#/patternProperties/^a/title", "instanceLocation": "/apple", "keyword": "title"}, {"annotation": ["apple"], "keywordLocation": "#/patternProperties", "instanceLocation": "", "keyword": "patternProperties"}, {"annotation": ["bar"], "keywordLocation": "#/unevaluatedProperties", "instanceLocation": "", "keyword": "unevaluatedProperties"}, {"annotation": "Unevaluated", "keywordLocation": "#/unevaluatedProperties/title", "instanceLocation": "/bar", "keyword": "title"}]}]}
{"seq": 29, "implementation": "java-networknt-json-schema-validator", "expected": [[{"location": "/foo", "keyword": "title", "expected": {"#/additionalProperties": "Evaluated"}}, {"location": "/bar", "keyword": "title", "expected": {"#/additionalProperties": "Evaluated"}}]], "results": [{"valid": true, "annotations": [{"annotation": "Evaluated", "keywordLocation": "#/additionalProperties/title", "instanceLocation": "/foo", "keyword": "title"}, {"annotation": "Evaluated", "keywordLocation": "#/additionalProperties/title", "instanceLocation": "/bar", "keyword": "title"}, {"annotation": ["foo", "bar"], "keywordLocation": "#/additionalProperties", "instanceLocation": "", "keyword": "additionalProperties"}, {"annotation": [], "keywordLocation": "#/unevaluatedProperties", "instanceLocation": "", "keyword": "unevaluatedProperties"}]}]}
{"seq": 30, "implementation": "java-networknt-json-schema-validator", "expected": [[{"location": "/foo", "keyword": "title", "expected": {"#/unevaluatedProperties": "Unevaluated"}}, {"location": "/bar", "keyword": "title", "expected": {"#/dependentSchemas/foo/properties/bar": "Evaluated"}}]], "results": [{"valid": true, "annotations": [{"annotation": "Evaluated", "keywordLocation": "#/dependentSchemas/foo/properties/bar/title", "instanceLocation": "/bar", "keyword": "title"}, {"annotation": ["bar"], "keywordLocation": "#/dependentSchemas/foo/properties", "instanceLocation": "", "keyword": "properties"}, {"annotation": ["foo"], "keywordLocation": "#/unevaluatedProperties", "instanceLocation": "", "keyword": "unevaluatedProperties"}, {"annotation": "Unevaluated", "keywordLocation": "#/unevaluatedProperties/title", "instanceLocation": "/foo", "keyword": "title"}]}]}
{"seq": 31, "implementation": "java-networknt-json-schema-validator", "expected": [[{"location": "/foo", "keyword": "title", "expected": {"#/then/properties/foo": "Then", "#/if/properties/foo": "If"}}, {"location": "/bar", "keyword": "title", "expected": {"#/unevaluatedProperties": "Unevaluated"}}], [{"location": "/foo", "keyword": "title", "expected": {"#/else/properties/foo": "Else"}}, {"location": "/bar", "keyword": "title", "expected": {"#/unevaluatedProperties": "Unevaluated"}}]], "results": [{"valid": true, "annotations": [{"annotation": "If", "keywordLocation": "#/if/properties/foo/title", "instanceLocation": "/foo", "keyword": "title"}, {"annotation": "Then", "keywordLocation": "#/then/properties/foo/title", "instanceLocation": "/foo", "keyword": "title"}, {"annotation": ["foo"], "keywordLocation": "#/if/properties", "instanceLocation": "", "keyword": "properties"}, {"annotation": ["foo"], "keywordLocation": "#/then/properties", "instanceLocation": "", "keyword": "properties"}, {"annotation": ["bar"], "keywordLocation": "#/unevaluatedProperties", "instanceLocation": "", "keyword": "unevaluatedProperties"}, {"annotation": "Unevaluated", "keywordLocation": "#/unevaluatedProperties/title", "instanceLocation": "/bar", "keyword": "title"}]}, {"valid": true, "annotations": [{"annotation": "Else", "keywordLocation": "#/else/properties/foo/title", "instanceLocation": "/foo", "keyword": "title"}, {"annotation": ["foo"], "keywordLocation": "#/else/properties", "instanceLocation": "", "keyword": "properties"}, {"annotation": ["bar"], "keywordLocation": "#/unevaluatedProperties", "instanceLocation": "", "keyword": "unevaluatedProperties"}, {"annotation": "Unevaluated", "keywordLocation": "#/unevaluatedProperties/title", "instanceLocation": "/bar", "keyword": "title"}]}]}
{"seq": 32, "implementation": "java-networknt-json-schema-validator", "expected": [[{"location": "/foo", "keyword": "title", "expected": {"#/allOf/0/properties/foo": "Evaluated"}}, {"location": "/bar", "keyword": "title", "expected": {"#/unevaluatedProperties": "Unevaluated"}}]], "results": [{"valid": true, "annotations": [{"annotation": "Evaluated", "keywordLocation": "#/allOf/0/properties/foo/title", "instanceLocation": "/foo", "keyword": "title"}, {"annotation": ["foo"], "keywordLocation": "#/allOf/0/properties", "instanceLocation": "", "keyword": "properties"}, {"annotation": ["bar"], "keywordLocation": "#/unevaluatedProperties", "instanceLocation": "", "keyword": "unevaluatedProperties"}, {"annotation": "Unevaluated", "keywordLocation": "#/unevaluatedProperties/title", "instanceLocation": "/bar", "keyword": "title"}]}]}
{"seq": 33, "implementation": "java-networknt-json-schema-validator", "expected": [[{"location": "/foo", "keyword": "title", "expected": {"#/anyOf/0/properties/foo": "Evaluated"}}, {"location": "/bar", "keyword": "title", "expected": {"#/unevaluatedProperties": "Unevaluated"}}]], "results": [{"valid": true, "annotations": [{"annotation": "Evaluated", "keywordLocation": "#/anyOf/0/properties/foo/title", "instanceLocation": "/foo", "keyword": "title"}, {"annotation": ["foo"], "keywordLocation": "#/anyOf/0/properties", "instanceLocation": "", "keyword": "properties"}, {"annotation": ["bar"], "keywordLocation": "#/unevaluatedProperties", "instanceLocation": "", "keyword": "unevaluatedProperties"}, {"annotation": "Unevaluated", "keywordLocation": "#/unevaluatedProperties/title", "instanceLocation": "/bar", "keyword": "title"}]}]}
{"seq": 34, "implementation": "java-networknt-json-schema-validator", "expected": [[{"location": "/foo", "keyword": "title", "expected": {"#/oneOf/0/properties/foo": "Evaluated"}}, {"location": "/bar", "keyword": "title", "expected": {"#/unevaluatedProperties": "Unevaluated"}}]], "results": [{"valid": true, "annotations": [{"annotation": "Evaluated", "keywordLocation": "#/oneOf/0/properties/foo/title", "instanceLocation": "/foo", "keyword": "title"}, {"annotation": ["foo"], "keywordLocation": "#/oneOf/0/properties", "instanceLocation": "", "keyword": "properties"}, {"annotation": ["bar"], "keywordLocation": "#/unevaluatedProperties", "instanceLocation": "", "keyword": "unevaluatedProperties"}, {"annotation": "Unevaluated", "keywordLocation": "#/unevaluatedProperties/title", "instanceLocation": "/bar", "keyword": "title"}]}]}
{"seq": 35, "implementation": "java-networknt-json-schema-validator", "expected": [[{"location": "/foo", "keyword": "title", "expected": {"#/unevaluatedProperties": "Unevaluated"}}, {"location": "/bar", "keyword": "title", "expected": {"#/unevaluatedProperties": "Unevaluated"}}]], "results": [{"valid": true, "annotations": [{"annotation": "Evaluated", "keywordLocation": "#/not/not/properties/foo/title", "instanceLocation": "/foo", "keyword": "title"}, {"annotation": "Unevaluated", "keywordLocation": "#/unevaluatedProperties/title", "instanceLocation": "/foo", "keyword": "title"}, {"annotation": ["foo", "bar"], "keywordLocation": "#/unevaluatedProperties", "instanceLocation": "", "keyword": "unevaluatedProperties"}, {"annotation": "Unevaluated", "keywordLocation": "#/unevaluatedProperties/title", "instanceLocation": "/bar", "keyword": "title"}]}]}
{"seq": 36, "implementation": "java-networknt-json-schema-validator", "expected": [[{"location": "/0", "keyword": "title", "expected": {"#/unevaluatedItems": "Unevaluated"}}, {"location": "/1", "keyword": "title", "expected": {"#/unevaluatedItems": "Unevaluated"}}]], "results": [{"valid": true, "annotations": [{"annotation": "Unevaluated", "keywordLocation": "#/unevaluatedItems/title", "instanceLocation": "/0", "keyword": "title"}, {"annotation": "Unevaluated", "keywordLocation": "#/unevaluatedItems/title", "instanceLocation": "/1", "keyword": "title"}, {"annotation": true, "keywordLocation": "#/unevaluatedItems", "instanceLocation": "", "keyword": "unevaluatedItems"}]}]}
{"seq": 37, "implementation": "java-networknt-json-schema-validator", "expected": [[{"location": "/0", "keyword": "title", "expected": {"#/prefixItems/0": "Evaluated"}}, {"location": "/1", "keyword": "title", "expected": {"#/unevaluatedItems": "Unevaluated"}}]], "results": [{"valid": true, "annotations": [{"annotation": "Evaluated", "keywordLocation": "#/prefixItems/0/title", "instanceLocation": "/0", "keyword": "title"}, {"annotation": 1, "keywordLocation": "#/prefixItems", "instanceLocation": "", "keyword": "prefixItems"}, {"annotation": true, "keywordLocation": "#/unevaluatedItems", "instanceLocation": "", "keyword": "unevaluatedItems"}, {"annotation": "Unevaluated", "keywordLocation": "#/unevaluatedItems/title", "instanceLocation": "/1", "keyword": "title"}]}]}
{"seq": 38, "implementation": "java-networknt-json-schema-validator", "expected": [[{"location": "/0", "keyword": "title", "expected": {"#/contains": "Evaluated"}}, {"location": "/1", "keyword": "title", "expected": {"#/unevaluatedItems": "Unevaluated"}}]], "results": [{"valid": true, "annotations": [{"annotation": "Evaluated", "keywordLocation": "#/contains/title", "instanceLocation": "/0", "keyword": "title"}, {"annotation": "Unevaluated", "keywordLocation": "#/unevaluatedItems/title", "instanceLocation": "/1", "keyword": "title"}, {"annotation": [0], "keywordLocation": "#/contains", "instanceLocation": "", "keyword": "contains"}, {"annotation": true, "keywordLocation": "#/unevaluatedItems", "instanceLocation": "", "keyword": "unevaluatedItems"}]}]}
{"seq": 39, "implementation": "java-networknt-json-schema-validator", "expected": [[{"location": "/0", "keyword": "title", "expected": {"#/then/prefixItems/0": "Then", "#/if/prefixItems/0": "If"}}, {"location": "/1", "keyword": "title", "expected": {"#/unevaluatedItems": "Unevaluated"}}], [{"location": "/0", "keyword": "title", "expected": {"#/else/prefixItems/0": "Else"}}, {"location": "/1", "keyword": "title", "expected": {"#/unevaluatedItems": "Unevaluated"}}]], "results": [{"valid": true, "annotations": [{"annotation": "If", "keywordLocation": "#/if/prefixItems/0/title", "instanceLocation": "/0", "keyword": "title"}, {"annotation": "Then", "keywordLocation": "#/then/prefixItems/0/title", "instanceLocation": "/0", "keyword": "title"}, {"annotation": 1, "keywordLocation": "#/if/prefixItems", "instanceLocation": "", "keyword": "prefixItems"}, {"annotation": 1, "keywordLocation": "#/then/prefixItems", "instanceLocation": "", "keyword": "prefixItems"}, {"annotation": true, "keywordLocation": "#/unevaluatedItems", "instanceLocation": "", "keyword": "unevaluatedItems"}, {"annotation": "Unevaluated", "keywordLocation": "#/unevaluatedItems/title", "instanceLocation": "/1", "keyword": "title"}]}, {"valid": true, "annotations": [{"annotation": "Else", "keywordLocation": "#/else/prefixItems/0/title", "instanceLocation": "/0", "keyword": "title"}, {"annotation": 1, "keywordLocation": "#/else/prefixItems", "instanceLocation": "", "keyword": "prefixItems"}, {"annotation": true, "keywordLocation": "#/unevaluatedItems", "instanceLocation": "", "keyword": "unevaluatedItems"}, {"annotation": "Unevaluated", "keywordLocation": "#/unevaluatedItems/title", "instanceLocation": "/1", "keyword": "title"}]}]}
{"seq": 40, "implementation": "java-networknt-json-schema-validator", "expected": [[{"location": "/0", "keyword": "title", "expected": {"#/allOf/0/prefixItems/0": "Evaluated"}}, {"location": "/1", "keyword": "title", "expected": {"#/unevaluatedItems": "Unevaluated"}}]], "results": [{"valid": true, "annotations": [{"annotation": "Evaluated", "keywordLocation": "#/allOf/0/prefixItems/0/title", "instanceLocation": "/0", "keyword": "title"}, {"annotation": 1, "keywordLocation": "#/allOf/0/prefixItems", "instanceLocation": "", "keyword": "prefixItems"}, {"annotation": true, "keywordLocation": "#/unevaluatedItems", "instanceLocation": "", "keyword": "unevaluatedItems"}, {"annotation": "Unevaluated", "keywordLocation": "#/unevaluatedItems/title", "instanceLocation": "/1", "keyword": "title"}]}]}
{"seq": 41, "implementation": "java-networknt-json-schema-validator", "expected": [[{"location": "/0", "keyword": "title", "expected": {"#/anyOf/0/prefixItems/0": "Evaluated"}}, {"location": "/1", "keyword": "title", "expected": {"#/unevaluatedItems": "Unevaluated"}}]], "results": [{"valid": true, "annotations": [{"annotation": "Evaluated", "keywordLocation": "#/anyOf/0/prefixItems/0/title", "instanceLocation": "/0", "keyword": "title"}, {"annotation": 1, "keywordLocation": "#/anyOf/0/prefixItems", "instanceLocation": "", "keyword": "prefixItems"}, {"annotation": true, "keywordLocation": "#/unevaluatedItems", "instanceLocation": "", "keyword": "unevaluatedItems"}, {"annotation": "Unevaluated", "keywordLocation": "#/unevaluatedItems/title", "instanceLocation": "/1", "keyword": "title"}]}]}
{"seq": 42, "implementation": "java-networknt-json-schema-validator", "expected": [[{"location": "/0", "keyword": "title", "expected": {"#/oneOf/0/prefixItems/0": "Evaluated"}}, {"location": "/1", "keyword": "title", "expected": {"#/unevaluatedItems": "Unevaluated"}}]], "results": [{"valid": true, "annotations": [{"annotation": "Evaluated", "keywordLocation": "#/oneOf/0/prefixItems/0/title", "instanceLocation": "/0", "keyword": "title"}, {"annotation": 1, "keywordLocation": "#/oneOf/0/prefixItems", "instanceLocation": "", "keyword": "prefixItems"}, {"annotation": true, "keywordLocation": "#/unevaluatedItems", "instanceLocation": "", "keyword": "unevaluatedItems"}, {"annotation": "Unevaluated", "keywordLocation": "#/unevaluatedItems/title", "instanceLocation": "/1", "keyword": "title"}]}]}
{"seq": 43, "implementation": "java-networknt-json-schema-validator", "expected": [[{"location": "/0", "keyword": "title", "expected": {"#/unevaluatedItems": "Unevaluated"}}, {"location": "/1", "keyword": "title", "expected": {"#/unevaluatedItems": "Unevaluated"}}]], "results": [{"valid": true, "annotations": [{"annotation": "Evaluated", "keywordLocation": "#/not/not/prefixItems/0/title", "instanceLocation": "/0", "keyword": "title"}, {"annotation": "Unevaluated", "keywordLocation": "#/unevaluatedItems/title", "instanceLocation": "/0", "keyword": "title"}, {"annotation": true, "keywordLocation": "#/unevaluatedItems", "instanceLocation": "", "keyword": "unevaluatedItems"}, {"annotation": "Unevaluated", "keywordLocation": "#/unevaluatedItems/title", "instanceLocation": "/1", "keyword": "title"}]}]}
{"seq": 44, "implementation": "java-networknt-json-schema-validator", "expected": [[{"location": "", "keyword": "x-unknownKeyword", "expected": {"#": "Foo"}}]], "results": [{"valid": true, "annotations": [{"annotation": "Foo", "keywordLocation": "#/x-unknownKeyword", "instanceLocation": "", "keyword": "x-unknownKeyword"}]}]}
{"did_fail_fast": false}
Ran 44 test cases.

[AgniveshChaubey]

Great! I assume the above updates doesnt break the existing bowtie suite command for validation. If so, I think we should move towards updating bowtie summary to make it smart enough to see which test suits's (annotation or validation) result is coming into stdin of summary command. Based on that it represents the comparison result. For that, the harness should always return either flag-based response or rich response (and not both). It would make it easy for the downstream commands to make decisions

[VIDIT45AGARWAL]

@AgniveshChaubey @Julian I have now updated the summary command -- automatically detects whether the incoming report is from an annotation or validation test suite.
For annotation reports, summary now shows a Pass / Fail table per test case, based on whether the expected annotation assertions matched the harness output -- this works across all three output formats (--format pretty , --format json, --format markdown
The test_type in the report header makes it easy for any downstream command to detect the reponse type (flag or rich), test_type field defaults to "validation", so existing reports without it continue to work as before