Skip to content

Commit 982cc97

Browse files
committed
Cleanup
1 parent 98e55bf commit 982cc97

2 files changed

Lines changed: 28 additions & 83 deletions

File tree

index.qmd

Lines changed: 26 additions & 62 deletions
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ abstract: |
1313

1414
# Introduction
1515

16-
Data validation is a crucial part of management of data qualitiy and interoperability. Validation is applied in many ways and contexts, for instance input forms and editors with visual feedback or schema languages with formal error reports. The diversity of use cases imply a variety of error results. No common standard exist to express error reports.^[A notable exception are formats from software development used in unit testing such as [JUnit XML](https://github.com/testmoapp/junitxml) and [Test Anything Protocol](https://testanything.org/).]
16+
Data validation is a crucial part of management of data qualitiy and interoperability. Validation is applied in many ways and contexts, for instance input forms and editors with visual feedback or schema languages with formal error reports. The diversity of use cases imply a variety of error results. No common standard exist to express error reports (except for unit testing formats such as [JUnit XML](https://github.com/testmoapp/junitxml) and [Test Anything Protocol](https://testanything.org/) with narrow use cases in software development).
1717

1818
The specification of **Data Validation Error Format** has two goals:
1919

@@ -26,6 +26,8 @@ Last but not least the format should help to better separate validation and pres
2626
The format is strictly limited to **errors** and **error positions**. Neither does it include other kinds of analysis results such as statistics and summaries of documents, nor does in include details about validation such as test cases, schema rules, and individual constraints. Errors can be linked to additional information with error types but the semantics of these types is out of the scope of this specification.
2727
:::
2828

29+
This document is managed in a revision control system at <https://github.com/gbv/validation-error-format>, including an [issue tracker](https://github.com/gbv/validation-error-format/issues).
30+
2931
## Overview
3032

3133
@fig-validation illustrates the validation process with core concepts used in this specification: a **validator** checks whether a **document** conforms to some requirements and returns a list of **errors** in return. Each error can refer to its location in the document via a **position**.
@@ -128,10 +130,8 @@ A similar document could be invalid on byte level. The following table illustrat
128130
+----------------+--------+--------+-----------------+--------+--------+--------+--------+--------+--------+
129131

130132

131-
132133
::: {#lst-3 lst-cap="Invalid JSON on multiple levels"}
133134

134-
135135
```{.json}
136136
[
137137
{
@@ -170,13 +170,14 @@ An **Error** is a JSON object with:
170170
Applications MAY use a default value for error messages.
171171

172172
- optional field `types` with an array of **error types**, each being a non-empty string.
173-
Error types can be used for grouping errors and they SHOULD be URIs. Repetitions of
174-
identical strings in the same array MUST be ignored.
173+
Error types can be used for grouping errors. Error types SHOULD be either local identifiers
174+
with same syntax as the name of a [dimension] or URIs ([RFC 3986]).
175175

176176
- optional field `level` with an **error level**, being one of the strings `error`, `warning`, or `info`.
177-
Application MUST NOT differentiate between error level `error` and no error level.
177+
Application MUST use default value `error` if this field is not given.
178178

179-
- optional field `position` with a [**position**](#positions). Applications MUST NOT differentiate between empty position and no position.
179+
- optional field `position` with a [**position**](#positions).
180+
Applications MUST NOT differentiate between no position and an empty position (an empty array or an empty JSON object).
180181

181182
::: {.callout-note}
182183
Language and localization of error messages is out of the scope of this specification.
@@ -262,7 +263,7 @@ A **locator map** is a JSON object that maps names of [**dimensions**](#sec-dime
262263
{ "line": "7", "char": "42" }
263264
```
264265

265-
A locator map is equivalent to an array of locators with key and value of the JSON object entries mapped to field `dimension` and `address` of each locator. An array of locators can be reduced to a locator map by dropping all nested errors and selecting only the first locator of each locator format.
266+
A locator map can be transformed to an equivalent array of [locators](#locators) with key and value of the JSON object entries mapped to field `dimension` and `address` of each locator. An array of locators can be reduced to a locator map by dropping all nested errors and selecting only the first locator of each locator format.
266267

267268
```{#lst-locator-map .json lst-cap="Equivalent array of locators"}
268269
[
@@ -277,15 +278,15 @@ Applications MAY restrict their support of Data Validation Error Format to posit
277278

278279
A **dimension** is a defined method to address elements of a document. Each dimension has:
279280

280-
- a unique **name**, being a string that start with lowercase letter `a` to `z`, optionally followed by a sequence of lowercase letters, digits `0` to `9` and/or `-`.
281+
- a unique **name**, being a string that start with lowercase letter `a` to `z`, optionally followed by a sequence of lowercase letters and digits `0` to `9`.
281282

282283
- a **locator format**, being a formal language of Unicode strings to encode references to parts of a document. The sets of strings of the language are called **addresses**.
283284

284285
- a **document model** matching the **locator format**.
285286

286287
Some dimensions imply a document model on addressed elements. For instance a [line number] addresses a character string and a [JSON Pointer] addresses a JSON value.
287288

288-
Applications SHOULD support the following dimensions:
289+
Applications SHOULD support the following dimensions. The [appendix](#sec-additional-dimensions) contains a non-normative note on additional dimensions not fully specified yet.
289290

290291
name | locator format | document model | element model
291292
:---------------|-------------------------|-----------------------------------------|------------------
@@ -303,8 +304,6 @@ name | locator format | document model
303304

304305
The **identifier** locator format with name `id` and locator values being arbitrary Unicode strings subsumes every other locator format because locators of same value refer to the same element. It can be used for any kind of formalized reference to elements of a document, but its main use case are record identifiers, unique names and similar identifier systems.
305306

306-
See [appendix](#sec-additional-dimensions) for more dimensions to be discussed.
307-
308307
:::{.callout-note}
309308
Dimensions are a subset of query languages. A dimension value locates to *one* element from a document. A query language (e.g. JSONPath, full XPath...) can locate a set of elements.
310309
:::
@@ -355,7 +354,7 @@ The **cell range** locator format with name `cells` is used to reference a selec
355354

356355
The **table selection** locator format with name `rfc7111` is used to reference a selection of connected cells in tabular data. It can reference [cell references](#cell-references), [cell ranges](#cell-ranges), full rows and full columns. The locator format follows the following grammar in ABNF syntax ([RFC 5234]) with case-sensitive string support ([RFC 7405]):
357356

358-
~~~
357+
```{.abnf}
359358
TableSelection = Cells / Rows / Columns
360359
Cells = %s"cell=" CellPosition [ "-" CellPosition ]
361360
Rows = %s"row=" RowPosition [ "-" RowPosition ]
@@ -364,7 +363,7 @@ CellPosition = RowPosition "," ColPosition
364363
ColPosition = %31-%39 *DIGIT
365364
Position = %31-%39 *DIGIT
366365
DIGIT = %x30-39
367-
~~~
366+
```
368367

369368
:::{.callout-note}
370369
Tabular selection locator is a proper subset of [RFC 7111] URI Fragment Identifier.
@@ -386,12 +385,12 @@ The **JSON Pointer** locator format with name `jsonpointer` is used to reference
386385

387386
The **XML Element Locator** format follows the following grammar in ABNF syntax ([RFC 5234]) with rule [QName] defined in Namespaces in XML 1.0 specification:
388387

389-
~~~
388+
```{.abnf}
390389
XMLElementLocator = "/" / 1*( "/" NodeTest )
391390
NodeTest = [ "@" ] QName [ "[ Position "]" ]
392391
Position = %31-%39 *DIGIT
393392
DIGIT = %x30-39 ; 0...9
394-
~~~
393+
```
395394

396395
Applications MAY append the string `[1]` to an XML Element Locator if it does not end with a `Position` to ensure that a single element is referenced.
397396

@@ -456,9 +455,6 @@ XML Element Locator is a proper subset of (X)Path Expressions from [XPath] speci
456455
- A. Wright, H. Andrews, B. Hutton, and G. Dennis: *JSON Schema Draft 2020-12*.
457456
June 2022. <https://json-schema.org/draft/2020-12/json-schema-core.html>
458457

459-
460-
- [RFC 9457](https://datatracker.ietf.org/doc/html/rfc9457) defines an extensible error format with fields `type` (`types` in this format), `status`, `title`, `detail` (`message` in this format), `instance` (`position` but as one URI). The example given in the specification is similar to this format.
461-
462458
# Appendices {.unnumbered}
463459

464460
## JSON Schemas {.unnumbered}
@@ -471,58 +467,26 @@ Error records can be validated with the following non-normative, non-extensive J
471467

472468
## Additional dimensions {#sec-additional-dimensions .unnumbered}
473469

474-
The following locator formats or standards are yet to be evaluated for its use as dimension:
470+
The following locator formats or standards are being considered for addition as [dimensions](#dimensions):
475471

476-
[fq]: https://github.com/wader/fq?tab=readme-ov-file#fq
472+
### fq
477473

478-
```{#lst-fq .json lst-cap="Error using fq to locate the internal timestamp of a file in a .gz archive"}
474+
The [fq] tool supports analysis of many [binary formats](https://github.com/wader/fq/blob/master/doc/formats.md) so it could be used to locate elements of binary data models. The locator value would be the format, followed by a colon, followed by a path expression:
475+
476+
```.json
479477
{
480-
"message": "Timestamp must not be in the future!",
478+
"message": "Timestamp of .gz archive must not be in the future",
481479
"position": {
482480
"fq": "gzip:.members[0].mtime"
483481
}
484482
}
485-
```
486-
487-
- [RFC 5147](https://tools.ietf.org/html/rfc5147) is not included because it mixes lines and characters, so the document model of the referenced element is not obvious. The handling of [line endings in RFC 5147](https://datatracker.ietf.org/doc/html/rfc5147#section-4.1) also depends on context.
488-
- `fq` for all binary formats supported by [fq] (see @lst-fq)
489-
- `RangeAddress` of [References in OpenDocument](https://docs.oasis-open.org/office/OpenDocument/v1.4/OpenDocument-v1.4-part4-formula.html#References)
490-
- IIIF (section in an image)
491-
- RDF graphs (every subset of an RDF graph is another RDF graph)
492-
- SHACL [focus node](https://www.w3.org/TR/shacl/#focusNodes) and [result path](https://www.w3.org/TR/shacl/#results-path) serialized in a subset of RDF Turtle
493-
- Subset of SQL SELECT statements
494-
- PDF highlighted text annotations
495-
- Some variant of [Property Graph Exchange Format (PG)](https://pg-format.github.io/) and/or CYPHER Query for property graphs
496-
- [PICA Path](https://format.gbv.de/query/picapath)
497-
- [MARCspec](https://format.gbv.de/query/marcspec)
498-
- ...
499-
500-
## Report format
501-
502-
The validation error format could be extened with a **validation report format** that wraps and/or links to validation errors with metadata such as:
503-
504-
- timestamp when validation has `started` and `finished` or `aborted`
505-
- number or processed documents
506-
- statistics such as number of errors
507-
508-
Errors may also be indexed by dimension and address, e.g.
509-
510-
~~~json
511483
{
512-
"errormap": {
513-
"line": {
514-
"7": [ ...errors in line 7 ]
515-
}
484+
"message": "Image width of PNG file is too large",
485+
"position": {
486+
"fq": "png:.chunks[0].width"
516487
}
517488
}
518-
~~~
519-
520-
521-
## Changes {.unnumbered}
522-
523-
This document is managed in a revision control system at <https://github.com/gbv/validation-error-format>, including an [issue tracker](https://github.com/gbv/validation-error-format/issues).
524-
525-
- **Version 0.1.0**
489+
```
526490

527-
Work in progress.
491+
[fq]: https://github.com/wader/fq?tab=readme-ov-file#fq
528492

schema.json

Lines changed: 2 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -35,7 +35,7 @@
3535
"description": "locator map",
3636
"type": "object",
3737
"patternProperties": {
38-
"^[a-z][a-z0-9-]*$": {
38+
"^[a-z][a-z0-9]*$": {
3939
"type": "string"
4040
}
4141
},
@@ -54,7 +54,7 @@
5454
"properties": {
5555
"dimension": {
5656
"type": "string",
57-
"pattern": "^[a-z][a-z0-9-]*$"
57+
"pattern": "^[a-z][a-z0-9]*$"
5858
},
5959
"address": {
6060
"type": "string"
@@ -68,24 +68,5 @@
6868
},
6969
"required": ["dimension", "address"]
7070
}
71-
},
72-
"errormap": {
73-
"type": "object",
74-
"patternProperties": {
75-
"^[a-z][a-z0-9-]*$": {
76-
"type": "object",
77-
"description": "dimension",
78-
"patternProperties": {
79-
"^.*$": {
80-
"type": "array",
81-
"item": {
82-
"$ref": "#"
83-
}
84-
}
85-
},
86-
"additionalProperties": false
87-
}
88-
},
89-
"additionalProperties": false
9071
}
9172
}

0 commit comments

Comments
 (0)