diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index c122554..354262c 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -7,36 +7,36 @@ Thank you for your interest in contributing to the Global Type System (GTS) Spec ### Prerequisites - **Git** for version control -- **JSON Schema validator** (optional, for testing schema examples) -- **Python 3.8+** (optional, for running reference implementations) +- **Node.js 20+ and `ajv-cli`** (optional, for validating JSON Schema examples as CI does) +- **Python 3.9–3.14** (optional, for working on the conformance test suite) +- **Docker** (optional, and recommended for running the conformance test suite) - **Your favorite editor** (VS Code with JSON Schema support recommended) ### Development Setup ```bash # Clone the repository -git clone +git clone https://github.com/GlobalTypeSystem/gts-spec.git cd gts-spec -# Optional: Install Python dependencies for reference implementations -pip install jsonschema - -# Optional: Install JSON Schema validator +# Optional: install the JSON Schema validator used by CI npm install -g ajv-cli ``` +The conformance tests run against an external GTS implementation over HTTP; this repository does not contain a reference implementation. See [`tests/README.md`](tests/README.md) for Docker and local Python setup. + ### Repository Layout ``` gts-spec/ -├── README.md # Main specification document +├── README.md # Normative specification ├── CONTRIBUTING.md # This file -├── LICENSE # License information -└── examples/ # Example GTS Types and instances - ├── events/ # Event-related examples - │ ├── types/ # GTS Type Schemas (JSON Schema documents) - │ └── instances/ # JSON instance examples - └── ... # Other domain examples +├── LICENSE # License +├── NOTICE # Attribution notices +├── adr/ # Architecture Decision Records (+ template.md) +├── examples/ # GTS Types and Instances in JSON, YAML, and TypeSpec +├── tests/ # Implementation-independent HTTP conformance tests +└── .github/workflows/ # CI and release workflows ``` ## Development Workflow @@ -57,16 +57,26 @@ Use descriptive branch names: Follow the specification standards and patterns described below. +#### Specification changes require an ADR + +Any proposal that adds normative behavior to the specification or intentionally changes existing normative behavior **MUST go through the Architecture Decision Record (ADR) process**. Start by copying the project-adapted MADR [`ADR template`](adr/template.md) to the next sequentially numbered file under [`adr/`](adr/) and fill in all applicable sections. The specification, tests, and examples must reference or implement the decision where applicable. + +Open the pull request with the ADR already at **`Status: Accepted`** — the decision is what is under review, so accepting the ADR *is* merging the pull request. There is no separate approval step and no `Proposed` state in the repository: an ADR on `main` is accepted by definition. A decision that is later replaced keeps its file and moves to `Status: Superseded`, with `Superseded by` pointing at the ADR that replaces it (and that ADR's `Supersedes` pointing back). The ADR may be reviewed in its own pull request or together with the resulting specification change; either way it must not be merged before the reviewers agree on the decision itself. + +A confirmed bug fix does not require a new ADR when it only restores behavior already established by the specification or an accepted ADR. If fixing the issue requires choosing new semantics, it is a specification change and therefore requires an ADR. + ### 3. Validate Your Changes ```bash -# Validate all schemas in a directory -ajv compile --strict=false -s "examples/events/types/*.schema.json" +# Validate the JSON Schema examples covered by CI +ajv compile -s "examples/*/types/*.schema.json" --strict=false -# Run Python reference implementation tests (if available) -python -m pytest tests/ +# Run the conformance tests against a GTS server already listening on port 8000 +python -m pytest tests/ --gts-base-url http://127.0.0.1:8000 ``` +Run the checks relevant to the files you changed. JSONC, YAML, TypeSpec, nested example directories, and unresolved `gts://` references may require their own format-aware validation in addition to the CI command above. For test-suite setup and targeted test invocations, follow [`tests/README.md`](tests/README.md). + ### 4. Commit Changes Follow a structured commit message format: @@ -113,8 +123,9 @@ Specification development guidelines: - Follow GTS identifier format rules strictly - Ensure all schemas use correct `$id` values -- Validate schemas against JSON Schema Draft 7 or later +- Declare the intended JSON Schema dialect with `$schema` and use keywords valid for that dialect; GTS is dialect-agnostic, while repository examples generally use Draft-07 for interoperability - Include both GTS Type Schemas (the canonical JSON definitions of types) and GTS Instance examples +- Keep normative specification changes, conformance tests, and examples aligned - Document any deviations or implementation-specific choices ## Releases @@ -139,8 +150,8 @@ When the specification moves to the next minor version (e.g. `0.11` → `0.12`), Releases are produced from `github.com/GlobalTypeSystem/gts-spec`. The workflow is restricted to that repository; pushing a tag from a fork has no effect. ```bash -git tag v0.11.3 -git push origin v0.11.3 +git tag vX.Y.Z +git push origin vX.Y.Z ``` The [`Release Tests Image`](.github/workflows/release-tests-image.yml) workflow: diff --git a/README.md b/README.md index e59bebd..db1ce64 100644 --- a/README.md +++ b/README.md @@ -1622,6 +1622,7 @@ Given an inheritance chain `S₀ → S₁ → … → Sₙ`: - **`null` at any depth deletes that key** from the effective object (per RFC 7396). The principal use case is to revert an ancestor-set value and let the trait-schema's `default` re-apply via the materialization step described in the Completeness check below — that is, a descendant writes `"": null` to "fall back to the schema default" without picking a specific value. If the deleted key is `required` and has no `default`, the completeness check (OP#13) fails for non-abstract types (the descendant must then either mark itself abstract or accept that "delete + required + no default" is an unresolvable contract). Authors who want `null` as an *intended* trait value cannot express it via this merge and must use a sentinel value documented as part of the trait shape. - Defaults declared in the effective trait-schema MUST be materialized into the effective traits object before the Completeness check runs (per ADR-0003): for every property declared in the effective trait-schema with a `default` and not present in the chain-merged object, the registry MUST substitute the default value. The Completeness check below (OP#13) operates on the resulting *materialized* effective traits object. - A publisher who wants a trait value to be **locked** across all descendants of a base type SHOULD declare `"const": ` for that property in `x-gts-traits-schema`. A descendant attempting to override the value will fail the standard JSON Schema validation that runs against the effective trait-schema (per the Completeness check below). No GTS-specific "immutability" rule is required — `const` is the mechanism. + - `const` constrains the **value** of a property, not its presence: per standard JSON Schema it asserts nothing when the property is absent. A publisher who additionally wants the trait to survive an RFC 7396 `null` deletion SHOULD either list the property in the containing object schema's `required` array (deletion then fails the Completeness check for non-abstract types) or declare a `default` equal to the `const` value (deletion then becomes a no-op, since materialization restores the value before validation). This is the general presence rule for traits — it is not specific to `const`: any optional inherited trait without a `default` can be removed by a descendant. - A descendant MAY redeclare a trait value with the same value the ancestor already declared (idempotent restatement). - See [`adr/0004-x-gts-traits-merge-strategy.md`](adr/0004-x-gts-traits-merge-strategy.md) for the rationale. diff --git a/adr/0001-derivation-form.md b/adr/0001-derivation-form.md index 5675559..7392a5d 100644 --- a/adr/0001-derivation-form.md +++ b/adr/0001-derivation-form.md @@ -56,6 +56,9 @@ This ADR does **not** make GTS a [JSON Schema Dialect](https://json-schema.org/l ## Considered Options +- Option 1 — Strict canonical form +- Option 2 — GTS Type Schema as a JSON Schema Extension (dialect-agnostic) *(chosen)* + Both options use the same running example. ### Running example used in this section @@ -207,7 +210,7 @@ P-OK validates; P-BAD fails. Syntactically valid and semantically compatible — ## Decision Outcome -Chosen: **Option 2 — GTS Type Schema as a JSON Schema Extension (dialect-agnostic).** +Chosen option: **Option 2 — GTS Type Schema as a JSON Schema Extension (dialect-agnostic).** Key normative consequences: diff --git a/adr/0002-x-gts-traits-schema.md b/adr/0002-x-gts-traits-schema.md index ad7847a..12720b6 100644 --- a/adr/0002-x-gts-traits-schema.md +++ b/adr/0002-x-gts-traits-schema.md @@ -41,7 +41,12 @@ ADR-0001 commits GTS to being an **extension of JSON Schema** (dialect-agnostic; ## Considered Options -Two options. Option 2 has two sub-variants (2A and 2B) which we evaluate inside Option 2. +- Option 1 — Trait-type as a separately-registered GTS Type (URI value) +- Option 2 — `x-gts-traits-schema` is a JSON Schema subschema, in two sub-variants: + - Option 2A — Subschema + implicit chain aggregation *(chosen)* + - Option 2B — Subschema + explicit composition by the author + +Option 2's sub-variants share the keyword's value space and differ only in who composes the declarations along the chain; they are evaluated inside Option 2 below. ### Option 1 — Trait-type as a separately-registered GTS Type (URI value) @@ -229,7 +234,7 @@ By construction, any value satisfying the effective trait-schema also satisfies ## Decision Outcome -Chosen: **Option 2A — `x-gts-traits-schema` is a JSON Schema subschema (object OR boolean); the registry composes declarations along the `$id` chain via `allOf`.** +Chosen option: **Option 2A — `x-gts-traits-schema` is a JSON Schema subschema (object OR boolean); the registry composes declarations along the `$id` chain via `allOf`.** Normative consequences: diff --git a/adr/0003-x-gts-traits-completeness.md b/adr/0003-x-gts-traits-completeness.md index 58e25ee..32c756a 100644 --- a/adr/0003-x-gts-traits-completeness.md +++ b/adr/0003-x-gts-traits-completeness.md @@ -92,6 +92,10 @@ ADR-0001 commits GTS to being an extension of JSON Schema (dialect-agnostic); AD ## Considered Options +- Option 1 — No spec-level enforcement (author's responsibility) +- Option 2 — Validate at instance creation time +- Option 3 — Validate at type registration (fail fast) *(chosen)* + Three top-level options on **when (or whether) to enforce completeness**. ### Option 1 — No spec-level enforcement (author's responsibility) @@ -223,7 +227,7 @@ Three ways to make the registration succeed: ## Decision Outcome -Chosen: **Option 3 — validate at type registration; non-abstract types MUST be complete.** +Chosen option: **Option 3 — validate at type registration; non-abstract types MUST be complete.** ### Definition of "complete" @@ -254,7 +258,7 @@ At the registration of type T: - **Non-abstract type with required-no-default and no explicit value.** Registration fails. Author resolves by (a) providing `x-gts-traits` value, (b) declaring a `default` in the schema, or (c) marking the type abstract. - **Final non-abstract type.** Same rule as any non-abstract — must be complete. No special bullet needed. -## Implications +### Implications - **OP#13 (Schema Traits Validation)** includes this rule; the operation explicitly conditions the completeness step on `x-gts-abstract != true`. - **§9.7.5** carries the normative wording of the completeness check in the "Validation" bullet block, expressed in terms of "non-abstract types." diff --git a/adr/0004-x-gts-traits-merge-strategy.md b/adr/0004-x-gts-traits-merge-strategy.md index e1ac1b2..345d251 100644 --- a/adr/0004-x-gts-traits-merge-strategy.md +++ b/adr/0004-x-gts-traits-merge-strategy.md @@ -71,6 +71,12 @@ The ADR commits to one of these choices. ## Considered Options +- Option 1 — No merge (each type self-contained) +- Option 2a — Shallow merge, descendant-last-wins +- Option 2b — Shallow merge, immutable-once-set (the rule in spec versions up to 0.11) +- Option 2c — RFC 7396 (JSON Merge Patch), descendant-last-wins *(chosen)* +- Option 3 — Per-property author-controlled merge + ### Option 1 — No merge (each type self-contained) Every type's `x-gts-traits` is read as the whole truth for that type alone. Ancestor declarations are ignored when computing the descendant's effective traits. (Defaults from the effective trait-schema still materialize per ADR-0003, since defaults live in the schema, not in `x-gts-traits`.) @@ -146,6 +152,8 @@ Effective traits object for `audit.v1~` under Option 2a: `{ "retention": "P90D", The effective traits object is computed by walking the chain root → leaf; once a key has been set by any layer, later layers MUST either omit it or repeat the same value. Conflicting redeclaration causes registration to fail. +**This was the normative rule in spec versions up to 0.11** (§9.7.5 "Immutable-once-set", together with "Immutable defaults" on the schema side). Rejecting it here is therefore a breaking change, not a newly filled gap — see the Backward compatibility note under Implications. + *Example.* Using the same base as Option 2a (`retention: "P30D"` declared on the base): ```jsonc @@ -283,7 +291,7 @@ Registration of the derived type fails at `retention` (locked). To succeed, the ## Decision Outcome -Chosen: **Option 2c — RFC 7396 (JSON Merge Patch) along the `$id` chain, descendant-last-wins at the leaf, with `const` in `x-gts-traits-schema` as the publisher's lock mechanism.** +Chosen option: **Option 2c — RFC 7396 (JSON Merge Patch) along the `$id` chain, descendant-last-wins at the leaf, with `const` in `x-gts-traits-schema` as the publisher's lock mechanism.** ### Normative consequences @@ -426,6 +434,11 @@ Effective traits after merge: `{ "indexed": false }` (descendant's value wins th - `"default": ` — the value is a **soft default**; descendants who don't set the property inherit it, descendants who do set it can override freely (last-wins). - neither — the property is open; descendants set or inherit values as they wish. +`const` locks the **value**, not the presence: JSON Schema asserts nothing about an absent property, so a descendant can still drop a `const`-constrained trait with an RFC 7396 `null` patch. This is the general presence rule for traits, not a `const`-specific gap — any optional inherited trait without a `default` is removable the same way. A publisher who wants presence guaranteed too picks one of: + +- add the property to the containing object schema's `required` array — the deletion then fails the ADR-0003 completeness check for non-abstract types (loud failure); +- declare a `default` equal to the `const` value — the deletion becomes a no-op, since materialization restores the value before validation (self-healing). + ### Worked example D — `null` to fall back to the trait-schema default The main motivation for the `null`-as-delete semantic from RFC 7396 is its clean interaction with ADR-0003 *materialization*: after the chain-merge produces the effective traits object, defaults declared in the effective trait-schema are applied for properties **not present** in that object. A descendant that writes `"": null` therefore removes the chain-merged value for that key, and the materialization step then fills it back in from the schema's `default` (if any). @@ -468,18 +481,20 @@ Effective traits: `{ "retention": "P7D" }`. The descendant successfully reverted - **Object-valued trait, descendant overrides one nested field.** Per Worked example B: ancestor's other nested fields are preserved; only the field the descendant restates is overridden. No need for descendants to restate the whole nested object. - **Array-valued trait, descendant declares a different array.** Arrays replace wholesale (per RFC 7396). If publishers need per-element composability, they should model the data as a keyed object rather than an array. - **`null` in `x-gts-traits` deletes the key.** Per RFC 7396, a leaf value of `null` removes that key from the effective object. The primary use case is letting ADR-0003 materialization re-apply the trait-schema's `default` for that key (see Worked example D). If the deleted key is `required` and has no `default`, the completeness check (ADR-0003) fails registration for non-abstract types. Authors who want `null` as an actual trait *value* cannot express it via this merge — they would need a sentinel (e.g., `"unset"`) and document it as part of the trait shape. +- **A `const`-constrained property is deleted with `null`.** If the property is also `required`, the completeness check fails and the lock holds. If it has a `default` equal to the `const` value, materialization restores it and the deletion is a no-op. If it is neither, its absence satisfies the schema — `const` asserts nothing about an absent property. - **Ancestor sets a value; descendant repeats the same value.** Permitted; both layers agree. - **No `x-gts-traits` anywhere in the chain.** Effective traits object is empty; ADR-0003's materialization fills in any defaults declared in the effective trait-schema; the completeness check (for non-abstract types) then validates the materialized object. - **`x-gts-traits` on an abstract base.** Carried forward into descendants exactly like any other layer; abstract status affects only completeness checking per ADR-0003, not merge. -## Implications +### Implications - **§9.7.5 ("Trait merge and validation semantics")** carries the normative wording of RFC 7396 merge along the `$id` chain and the `const`-based lock mechanism. - **ADR-0003** stays correct as written; the "chain-merged effective traits object" referenced there is now formally defined as the result of applying each layer's `x-gts-traits` as a JSON Merge Patch (RFC 7396) to the chain-merged object so far, root → leaf. - **OP#13 description (§9.7)** is unaffected; it speaks generically of "chain-merged" values. - **§9.11.4 (modifiers ↔ traits)** is unaffected; completeness keying on `x-gts-abstract` is independent of merge policy. - **Reference implementations (gts-go, gts-rust)** must implement RFC 7396 merge along the chain. Available implementations exist in both ecosystems. The registry MUST NOT enforce a "different value MUST fail" rule on its own — it relies on standard JSON Schema validation against the effective trait-schema (which catches `const` violations naturally). -- **Conformance test suite** should exercise: (a) descendant overrides a top-level scalar — succeeds (last-wins); (b) descendant overrides one field of a nested-object trait — other nested fields preserved; (c) descendant overrides an array-valued trait — array replaces wholesale; (d) descendant writes `null` at a leaf — the key is removed; (e) descendant repeats the same value — succeeds (idempotent); (f) publisher locks via `const`, descendant attempts override — fails JSON Schema validation; (g) chain with three layers; middle layer overrides base; leaf overrides middle. +- **Conformance test suite** should exercise: (a) descendant overrides a top-level scalar — succeeds (last-wins); (b) descendant overrides one field of a nested-object trait — other nested fields preserved; (c) descendant overrides an array-valued trait — array replaces wholesale; (d) descendant writes `null` at a leaf — the key is removed; (e) descendant repeats the same value — succeeds (idempotent); (f) publisher locks via `const`, descendant attempts override — fails JSON Schema validation; (g) publisher locks via `const` and `required`, descendant attempts to delete the property with `null` — fails the completeness check; (h) chain with three layers; middle layer overrides base; leaf overrides middle. +- **Backward compatibility: breaking** (reflected in the README changelog entry for 0.12). This decision does not fill a gap — it reverses a prior normative rule. Spec versions up to 0.11 required "immutable-once-set" for trait values (a descendant supplying a different value for a key already set by an ancestor MUST fail validation) plus "immutable defaults" (a descendant MUST NOT redeclare an ancestor's `default`) — that is, Option 2b above, which this ADR rejects. Under RFC 7396 last-wins, a chain that MUST have been rejected now registers successfully, and locking becomes opt-in for the publisher via `const` instead of implicit. Implementations MUST remove the immutable-once-set and immutable-defaults checks; conformance tests asserting the old rejection MUST be rewritten. Registries that relied on the implicit guarantee SHOULD audit published base types and add `const` where a fixed trait value was intended. ## Pros and Cons of the Options diff --git a/adr/template.md b/adr/template.md new file mode 100644 index 0000000..5225795 --- /dev/null +++ b/adr/template.md @@ -0,0 +1,59 @@ +# ADR-NNNN: Short decision title — chosen outcome + +- **Status:** Accepted +- **Date:** YYYY-MM-DD +- **Deciders:** Names or roles responsible for the decision +- **Consulted:** Names or roles consulted during the decision +- **Supersedes:** ADR-NNNN or — +- **Superseded by:** ADR-NNNN or — + +## Context and Problem Statement + + + +## Decision Drivers + + + +- Decision driver + +## Considered Options + + + +- Option 1 — Short name +- Option 2 — Short name *(chosen)* + +### Option 1 — Short name + + + +### Option 2 — Short name + + + +## Decision Outcome + +Chosen option: "Option N", because it best satisfies the decision drivers described above. + +### Implications + + + +- Consequence or required follow-up + +## Pros and Cons of the Options + +### Option 1 — Short name + +- **+** Benefit +- **−** Drawback + +### Option 2 — Short name + +- **+** Benefit +- **−** Drawback + +## More Information + + diff --git a/tests/test_op13_schema_traits_validation.py b/tests/test_op13_schema_traits_validation.py index 1e3e847..40c540f 100644 --- a/tests/test_op13_schema_traits_validation.py +++ b/tests/test_op13_schema_traits_validation.py @@ -3391,6 +3391,56 @@ def test_start(self): ] +class TestCaseOp13_Merge_ConstLock_NullDeleteFails(HttpRunner): + """ADR-0004 §"Conformance test suite" (g): null cannot delete a required lock. + + RFC 7396 removes `indexed` from the merged traits object, but the effective + trait-schema also requires the property. The materialized object therefore + fails OP#13 instead of bypassing the const constraint. + """ + + config = Config("OP#13 ADR-0004: const lock rejects null delete").base_url( + get_gts_base_url() + ) + + def test_start(self): + super().test_start() + + teststeps = [ + _register( + "gts://gts.x.test13.mconstdel.event.v1~", + { + "type": "object", + "x-gts-traits-schema": { + "type": "object", + "properties": { + "indexed": {"type": "boolean", "const": True}, + }, + "required": ["indexed"], + }, + "x-gts-traits": {"indexed": True}, + "required": ["id"], + "properties": {"id": {"type": "string"}}, + }, + "register base with const-and-required locked indexed=true", + ), + _register_derived( + "gts://gts.x.test13.mconstdel.event.v1~x.test13._.kid.v1~", + "gts://gts.x.test13.mconstdel.event.v1~", + { + "type": "object", + "x-gts-traits": {"indexed": None}, + }, + "register descendant trying to delete indexed", + ), + _validate_type_schema( + "gts.x.test13.mconstdel.event.v1~x.test13._.kid.v1~", + False, + "validate descendant - required prevents const-lock deletion", + ), + ] + + class TestCaseOp13_Merge_ConstLock_IdempotentRestatementOk(HttpRunner): """ADR-0004: descendant restates const-locked value. Passes."""