feat(openlabel): add structural LinkML schema for ASAM v1 JSON format

[jdsika]

Description

Adds a structural LinkML schema (openlabel-v2-schema.yaml) that models the
ASAM OpenLABEL v1.0.0 scenario tagging JSON file format and generates a JSON
Schema that validates both the file structure and the tag vocabulary —
something the original ASAM schema does not do (it accepts any string as
tag.type).

This complements the existing semantic model (openlabel-v2.yaml, which already
generates OWL/SHACL/JSON-LD context). The two models share a single source of
truth via TagTypeEnum, the list of valid tag.type values derived from the
ontology vocabulary.

The schema and vocabulary were hardened against an adversarial audit that
validated outputs directly against the authoritative ASAM JSON Schema and
scenario tagging ontology (see "Audit hardening" below).

Architecture

Schema	Purpose	Generates
openlabel-v2.yaml	Semantic model (tag vocabulary)	OWL, SHACL, JSON-LD context
openlabel-v2-schema.yaml	Structural model (file format)	JSON Schema for the v1 format

The bridge is TagTypeEnum — 256 valid tag.type values derived from the
ontology's classes, slots, and enum values. Changes to the ontology flow into the
JSON Schema when the enum is regenerated (scripts/sync_tag_type_enum.py, wired
as a pre-commit hook).

Changes

New files

• linkml/openlabel-v2/openlabel-v2-schema.yaml — structural model (12 classes,
  5 enums) of the openlabel root, metadata, ontologies, tags, tag_data,
  and the generic boolean/num/text/vec data containers.
• artifacts/openlabel-v2/openlabel-v2.schema.json — generated JSON Schema artifact.
• linkml/openlabel-v2/SCHEMA_MODELING.md — architecture, scope, and a
  functional-equivalence comparison against the ASAM schema.
• scripts/sync_tag_type_enum.py — regenerates TagTypeEnum from the semantic
  model (all openlabel_v2 tag classes + admin tags + enum/boolean slots).
• scripts/convert_openlabel_v1_to_v2.py — converts plain ASAM v1 tagging JSON
  into JSON-LD using the v2 context (for SHACL validation / semantic search).
• tests/unit/test_openlabel_schema.py — 48 pytest tests (generation, validation,
  cross-schema equivalence, vocabulary faithfulness, sync, converter).

Modified files

• linkml/openlabel-v2/openlabel-v2.yaml — added 15 intermediate ODD/Behaviour
  category classes and 10 mid-level enum values (see Audit hardening, B2); all new
  tags carry ISO 34503:2023 comments: and v1 PAS 1883 notes:, matching the
  model's ISO-migration convention. Also adds domain: QuantitativeValue to the
  minValue/maxValue slots and the combined TrafficAgentTypeEnum range on
  trafficAgentTypeValue (see "JSON-LD data simplification" below).
• tests/data/openlabel-v2/** — valid and invalid instances simplified (nested
  @type removed, @type: QuantitativeValue dropped, bare enum terms); fail10/
  fail12 .expected regenerated; new fail13 regression test + catalog entry.
• artifacts/openlabel-v2/* — regenerated OWL, SHACL, JSON-LD context, JSON
  Schema, and PROPERTIES.md.
• Makefile — new gen-json-schema step (auto-detects a *-schema.yaml model)
  using --deterministic --indent 3 and the same trailing-newline normalization
  as the OWL/SHACL/context targets.
• .pre-commit-config.yaml — sync-tag-type-enum hook; registry/properties/readme
  triggers extended to schema.json.
• submodules/asam-openx-standards — bumped to include the extracted ASAM
  OpenLABEL v1.0.0 spec/ontology/JSON-schema used as the normative reference.

Audit hardening

The original structural schema was reviewed against the ASAM OpenLABEL v1.0.0
spec, JSON Schema, and tagging ontology. The following findings were fixed:

ID	Severity	Issue	Fix
B1	Blocking	vec.val was string-only; rejected the spec's own numeric ranges/sets ([3.4, 3.7], [2, 3])	standard any_of[float, string] (clean, draft-independent anyOf)
B2	Blocking	Vocabulary covered 227/276 names; omitted 49 real ASAM tags incl. RoundaboutLarge ("large roundabout"), SceneryJunction, all mid-hierarchy classes	Added 15 category classes + 10 enum values; enum now 256 values covering all 243 ASAM tag classes (0 missing)
B3	Blocking	"Functional equivalence" was asserted but never tested against ASAM; one vec test masked B1 with string values	Tests now validate spec examples against BOTH the ASAM and LinkML schemas; numeric example restored
M1	Major	tag_data string form (ASAM oneOf[object,string]) was dropped	any_of[TagData, string]
M2	Major	Under-strict: schema_version, boolean/text type unconstrained; fabricated set vec type	const: "1.0.0", ValueOnlyTypeEnum, removed set
M4	Major	non-deterministic JSON Schema generation	gen-json-schema --deterministic --indent 3 (consistent with OWL/SHACL/context; LinkML is pinned, so metamodel_version is stable)
M3	Major	Equivalence/stricter claims overstated	Claims corrected and scoped to scenario-tagging files in SCHEMA_MODELING.md

ontology_uid is intentionally kept required on tags: every scenario tag
references its ontology (spec 8.2.1), all spec examples include it, and making it
optional triggers a LinkML simple-dict shorthand (bare-string tags) that ASAM
forbids.

Functional equivalence (verified)

11 spec-valid chapter-8 examples validate against both the authoritative ASAM
JSON Schema and the LinkML-generated schema (minimal, RoundaboutLarge, numeric
vec range/set, string vec set, num value/min, tag_data string, ontology-as-
string, boundary include, admin text). The LinkML schema is stricter only where
it adds value: it rejects unknown tag.type, non-1.0.0 schema_version, the
fabricated vec set type, bare-string tags, and non-number/string vec.val
items — all of which the ASAM schema accepts. The vec.val number|string
constraint was verified to hold under both JSON Schema draft-07 and 2019-09.

Refinement proof (programmatic)

The equivalence claims above are additionally backed by a falsifiable,
programmatic proof that the LinkML-generated JSON Schema faithfully re-models
the normative ASAM OpenLABEL v1.0.0 JSON Schema within the scenario-tagging
scope — plus a reusable harness for any future JSON-Schema → LinkML migration.

Full JSON-Schema equivalence is undecidable, so instead of cherry-picked
examples we prove a refinement relation:

L = A|τ refined by Δ, relaxed by Λ

• A — normative ASAM schema; A|τ — its tagging projection, derived by $ref
  reachability from metadata/ontologies/tags (10 in-scope defs, 36 out of
  scope — computed, not asserted).
• L — the LinkML-generated schema.
• Δ — enumerated intended strengthenings (256-value vocabulary enum, required
  ontology_uid, closed objects, schema_version const).
• Λ — enumerated, disclosed relaxations where L is knowingly looser.

Three pillars, all in scripts/schema_refinement_prover.py:

1. Scope projection — decidable reachability partition.
2. Structural gap table — every (def, aspect) classified
   EQUIVALENT / REFINEMENT / OUT_OF_SCOPE / LOOSER / UNMAPPED, each justified.
3. Differential oracle — a mutation + property-based corpus
   (hypothesis-jsonschema) validated against both schemas; the soundness gate
   fails on a single instance L accepts that A rejects for a reason outside Λ.

Closing the gaps the oracle found

The oracle surfaced two real relaxations the example-based tests missed. Both
are now closed using two jsonschemagen features upstreamed to the pinned
linkml fork (ASCS-eV/linkml #15
--include-null/--no-include-null,
#16 propertyNames from
inlined-dict key-slot constraints), and are regression-guarded: they are
intentionally not listed in allow_looseness, so the gate fails if either
ever reappears.

• null_optional — LinkML codegen emitted optional fields as
  type: ["t","null"], so L accepted explicit null where ASAM forbids it.
  Closed natively with --no-include-null, scoped to this domain via
  linkml/openlabel-v2/jsonschema.genopts.
• key_pattern — ASAM restricts dict keys (tag/ontology uids) to
  numeric/UUID patterns; L accepted any key. Closed by pattern on the uid
  identifier slots (emitted as propertyNames) plus --closed
  (root additionalProperties: false).

A model-correctness fix fell out of the same pass: tag.resource_uid was
modeled as a string id-ref, but ASAM defines it as an id → string map. It is
remodeled as an inlined simple-dict (ResourceUid: uid +
identifier_in_resource) with a patterned key.

The single remaining declared relaxation is Λ = {simple_dict_object_form}:
a LinkML inlined simple-dict always also accepts the object form
{"identifier_in_resource": ...} for its value, where ASAM allows only the
bare string. This is a LinkML expressivity limit (an inlined simple-dict cannot
be restricted to scalar-only values), not a model defect; it affects only the
optional resource_uid value form, is disclosed in proof_spec.yaml, and is
recognized as a category by the prover.

Result: linkml/openlabel-v2/REFINEMENT_PROOF.md — PROVEN (sound refinement
modulo declared relaxations), Λ = {simple_dict_object_form}, 0 undisclosed
looseness, 0 structural LOOSER rows.

CI hardening: the dev linkml dependency is pinned to the exact submodule
commit (97e73d0f) rather than a branch, so CI can't install a newer linkml
than the submodule and regenerate divergent artifacts. A slow pytest marker
is registered for the proof suite (pytest -m 'not slow' for the fast lane).

JSON-LD data simplification via RDFS inference

Building on the recent blank-node RDFS-inference support, the openlabel-v2 test
data is simplified so instances carry only the information a real producer would
write — nested container types are now inferred rather than declared.

• gen-owl --no-use-native-uris (Makefile): emits QuantitativeValue,
  minValue/maxValue and hasLowerBound/hasUpperBound with their declared
  mapped IRIs (schema: / cmns-q:) instead of native openlabel_v2: IRIs, so
  the shared context can resolve the QuantitativeValue value type. Scoped to
  openlabel-v2 (the only LinkML domain).
• Nested @type removal: redundant @type on AdminTag/Behaviour/Odd/
  RoadUser container blank nodes is dropped from every valid and invalid
  instance; the type is recovered from each parent property's rdfs:range under
  RDFS inference.
• domain: QuantitativeValue on the minValue/maxValue slots makes
  gen-owl emit schema:minValue/schema:maxValue rdfs:domain schema:QuantitativeValue, so RDFS infers the QuantitativeValue type from the
  mere presence of a range. The redundant @type: QuantitativeValue is removed
  from every valid node that carries a range; fail09 (no min/max) intentionally
  keeps its explicit @type.
• TrafficAgentTypeEnum: trafficAgentTypeValue moves from
  any_of: [RoadUserHumanEnum, RoadUserVehicleEnum] to a single combined enum.
  A single enum range restores the @type: @vocab context coercion (LinkML drops
  it for any_of slots) and yields one sh:in, so the data replaces verbose
  [{"@id": "openlabel_v2:VehicleCar"}, ...] objects with bare terms
  ["VehicleCar", "HumanPedestrian"], consistent with every other enum slot.
• sync_tag_type_enum.py dedup: the structural TagTypeEnum is a set of
  unique tag-type names, but the combined TrafficAgentTypeEnum repeats the
  RoadUser*Enum values. Each value is now emitted once — otherwise
  gen-json-schema aborts on a duplicate key. The synced
  openlabel-v2-schema.yaml gains only the trafficAgentTypeValue category slot
  and schema.json lists it as a valid tag type.
• New regression test fail13_quantitativevalue_inferred_missing_max: an
  untyped QuantitativeValue with minValue only (no @type, no maxValue). It
  proves the type is recovered purely by RDFS inference and that the required
  maxValue constraint still fires — the .expected shows exactly
  maxValue (QuantitativeValue), not a "not a QuantitativeValue" error, so a
  regression in inference would change the message and fail the test.

Versioning & Compatibility

• Patch
• Minor (New functionality that is backward-compatible)
• Major

Type of change

• New type (non-breaking change which adds a type)
• Change
• Breaking change

How Has This Been Tested?

• python scripts/schema_refinement_prover.py --spec linkml/openlabel-v2/proof_spec.yaml
  → exit 0, verdict PROVEN, Λ = {simple_dict_object_form}.
• pytest tests/unit/test_schema_refinement.py — 10/10 passed (soundness
  gate, coverage gate, scope-projection pins, bounded-looseness check asserting
  key_pattern/null_optional stay closed).
• pytest tests/unit/test_openlabel_schema.py -v — 48/48 passed (schema
  generation, ASAM/LinkML cross-validation, vocabulary faithfulness covering all
  243 ASAM tag classes, sync correctness, v1→v2 converter).
• python -m src.tools.validators.validation_suite --domain openlabel-v2 — SHACL
  conformance passes with nested types removed (valid instances conform via RDFS
  inference); all 13 invalid fixtures (incl. the new
  fail13_quantitativevalue_inferred_missing_max) fail as expected against their
  .expected files.
• make generate DOMAIN=openlabel-v2 is idempotent (CI "verify no changes");
  regeneration reproduces the committed openlabel-v2.schema.json byte-identically.
• ruff check clean on changed Python.

Checklist

• My code follows the modelling guidelines of this project
• I have performed a self-review of my own changes
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes