Skip to content

Commit a42cd6c

Browse files
docs(adr): 0003-drift-categories — input, distance, threshold per category (#91)
Closes #48. `src/tier1/drift.rs::DriftCategory` lists eight cross-modal drift categories, but the names alone don't tell a detector implementer what to compute. Without the (input, distance, threshold) triple fixed in writing, two detectors of "Temporal drift" can disagree about whether the same input pair is drifted. ADR-0003 fixes the triple for each of the eight categories: - **Temporal**: version skew on temporal_versions rows (`|Δv| / max(v_a, v_b)`), threshold 0.1. Pure ordering math — V-L1-E2 picks this up first. - **Structural**: schema-fingerprint Jaccard, threshold 0.05. - **Semantic**: cosine of sentence embeddings, threshold 0.20. - **Statistical**: 1-Wasserstein normalised by field range, threshold 0.15. - **Referential**: edge-set Jaccard, threshold 0.10. - **Provenance**: provenance-chain-tip mismatch or LCP-based fractional, threshold 0.05. - **Spatial**: Haversine normalised by half-earth-circumference, threshold 0.0001 (~2 km). - **Embedding**: cosine between stored vector and recomputed vector, threshold 0.10. Every category produces `[0, 1]` so the aggregator (`DriftReport::overall_score`) can take a weighted sum without re-normalising. Each entry has one worked example for the distance function. Three open questions called out for follow-up ADRs (embedding model choice, per-field scale for unbounded numerics, drift-report retention). Cross-references: - V-L1-E2 / issue #49 (Temporal detector, first impl) - V-L1-B1 / `docs/theory/provenance-threat-model.adoc` - `src/tier1/drift.rs::DriftCategory` Doc-only; no code changes. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
1 parent 0a509a9 commit a42cd6c

1 file changed

Lines changed: 244 additions & 0 deletions

File tree

Lines changed: 244 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,244 @@
1+
= Architecture Decision Record: 0003-drift-categories
2+
<!-- SPDX-License-Identifier: PMPL-1.0-or-later -->
3+
<!-- Copyright (c) 2026 Jonathan D.A. Jewell (hyperpolymath) <j.d.a.jewell@open.ac.uk> -->
4+
5+
# 3. Eight drift categories: input, distance, threshold
6+
7+
Date: 2026-05-14
8+
9+
## Status
10+
11+
Accepted
12+
13+
## Context
14+
15+
`src/tier1/drift.rs::DriftCategory` enumerates eight cross-modal drift
16+
categories. The names are evocative but unimplementable: a detector
17+
needs three things per category — what inputs to compare, how to
18+
score the comparison, where to draw the line. Without that triple
19+
fixed in writing, every detector implementation is guessing, and
20+
two implementations of "Temporal drift" can disagree about whether
21+
a given pair of inputs is drifted.
22+
23+
This ADR pins the triple for each of the eight categories. It is
24+
binding for V-L1-E2 (temporal detector, the cheapest first) and
25+
every subsequent V-L1-E* detector.
26+
27+
## Decision
28+
29+
Each category is defined by:
30+
31+
* **Input type** — the two artifacts being compared.
32+
* **Distance function** — total order on `[0.0, 1.0]` where `0.0`
33+
means "no drift" and `1.0` means "maximally drifted". Composability
34+
matters: an aggregator can take a weighted sum of per-category
35+
scores and still produce a value in the same range.
36+
* **Threshold range** — the default cutoff above which an entity is
37+
reported as drifted in this category. Configurable per deployment
38+
via the `--threshold` flag on `verisimiser drift`.
39+
40+
### 3.1 Temporal (cheapest; ship first via V-L1-E2)
41+
42+
* **Input**: two `(version, valid_from, valid_to)` triples from
43+
`verisimdb_temporal_versions` for the same entity across two
44+
modalities (e.g. the graph copy and the document copy).
45+
* **Distance**:
46+
+
47+
[source]
48+
----
49+
score = clamp01(|v_a.version - v_b.version| / max(v_a.version, v_b.version, 1))
50+
----
51+
+
52+
Pure ordering math; no payload comparison.
53+
* **Threshold default**: `0.1` (within 10% of each other).
54+
* **Example**: `(v=5, …)` vs `(v=4, …)``|5-4|/5 = 0.2` → drifted.
55+
`(v=10, …)` vs `(v=9, …)``|10-9|/10 = 0.1` → at threshold.
56+
57+
### 3.2 Structural
58+
59+
* **Input**: schema fingerprints of two modalities at the same
60+
point in time. Fingerprint = sorted list of `(field, type)` pairs.
61+
* **Distance**:
62+
+
63+
[source]
64+
----
65+
score = 1 - jaccard(fp_a, fp_b)
66+
= 1 - |fp_a ∩ fp_b| / |fp_a ∪ fp_b|
67+
----
68+
* **Threshold default**: `0.05`. Schemas should be near-identical;
69+
any divergence is a red flag.
70+
* **Example**: graph copy has `{(id, int), (title, text)}`, document
71+
copy has `{(id, int), (title, text), (body, text)}`. Intersection
72+
size 2, union size 3 → `1 - 2/3 = 0.33` → strongly drifted.
73+
74+
### 3.3 Semantic
75+
76+
* **Input**: free-text descriptions or labels stored alongside the
77+
entity in two modalities. (e.g. graph node label vs document title.)
78+
* **Distance**:
79+
+
80+
[source]
81+
----
82+
score = 1 - cosine(embed(text_a), embed(text_b))
83+
where embed() = sentence embedding (model TBD; pluggable)
84+
----
85+
+
86+
Cosine of two unit vectors is in `[-1, 1]`; we clamp to `[0, 1]`
87+
by `(1 - cos) / 2` if needed, but in practice sentence embeddings
88+
keep cosine in `[0, 1]`.
89+
* **Threshold default**: `0.20`.
90+
* **Example**: `"Big Launch Today"` vs `"Today: Big Launch"`
91+
cosine ≈ 0.98 → score 0.02 → not drifted.
92+
`"Big Launch Today"` vs `"Q3 earnings missed"`
93+
cosine ≈ 0.15 → score 0.85 → strongly drifted.
94+
95+
### 3.4 Statistical
96+
97+
* **Input**: two distributions over the same numeric / categorical
98+
field, sampled from two modalities at the same point in time.
99+
Continuous → KDE; discrete → multinomial histogram.
100+
* **Distance**:
101+
+
102+
[source]
103+
----
104+
score = wasserstein_1(dist_a, dist_b) / range(field)
105+
# 1-Wasserstein, also known as Earth-Mover's Distance.
106+
# Dividing by field range normalises to [0,1] for bounded fields.
107+
----
108+
+
109+
For unbounded fields, use `tanh(wasserstein_1 / scale)` with scale
110+
fixed in the deployment config.
111+
* **Threshold default**: `0.15`.
112+
* **Example**: vector modality reports a feature mean of `0.5`,
113+
tensor modality reports `0.7`, field range `[0, 1]`. EMD ≈ 0.2,
114+
normalised → 0.2 → drifted.
115+
116+
### 3.5 Referential
117+
118+
* **Input**: set of foreign-key / cross-reference edges in modality A
119+
versus modality B for the same entity. (e.g. graph adjacency vs
120+
document cross-references.)
121+
* **Distance**:
122+
+
123+
[source]
124+
----
125+
score = 1 - jaccard(edges_a, edges_b)
126+
----
127+
+
128+
Same shape as Structural but applied to the *edge* set rather than
129+
the *attribute* set.
130+
* **Threshold default**: `0.10`.
131+
* **Example**: graph says `entity-A → {entity-B, entity-C}`, document
132+
says cross-references `{entity-B}`. Jaccard `1/2 = 0.5`. Score
133+
`1 - 0.5 = 0.5` → strongly drifted.
134+
135+
### 3.6 Provenance
136+
137+
* **Input**: provenance chain tip hash in modality A versus
138+
modality B for the same entity. (Each modality may have its own
139+
hash chain in the sidecar; this category reports cross-chain drift.)
140+
* **Distance**:
141+
+
142+
[source]
143+
----
144+
score = if tip_a == tip_b { 0.0 }
145+
else { 1.0 - (longest_common_prefix(chain_a, chain_b)
146+
/ max(len(chain_a), len(chain_b))) }
147+
----
148+
+
149+
Binary in the easy case, fractional when chains share a prefix but
150+
have diverged.
151+
* **Threshold default**: `0.05`. Provenance divergence is almost
152+
always a real problem.
153+
* **Example**: chain A has 10 entries, chain B has 10 entries, they
154+
share the first 7. Score `1 - 7/10 = 0.3` → drifted.
155+
156+
### 3.7 Spatial
157+
158+
* **Input**: geographic / coordinate representations of an entity
159+
in two modalities. Either `(lat, lon)` pairs or a richer geometry.
160+
* **Distance**:
161+
+
162+
[source]
163+
----
164+
score = haversine(coord_a, coord_b) / max_radius
165+
# max_radius default: half-earth-circumference ≈ 20000 km
166+
# producing the same [0,1] range as the other categories.
167+
----
168+
+
169+
Geometry-richer inputs reduce to centroids before measurement;
170+
shape mismatch is reported separately under Structural.
171+
* **Threshold default**: `0.0001` (about 2 km).
172+
* **Example**: graph stores `(51.5074, -0.1278)` (London), document
173+
stores `(51.5081, -0.1290)`. Haversine ≈ 0.13 km. Normalised
174+
`0.13 / 20000 ≈ 6.5e-6` → not drifted.
175+
176+
### 3.8 Embedding
177+
178+
* **Input**: vector embedding stored in the vector modality versus
179+
a freshly recomputed embedding from the document-modality source
180+
text. Catches stale vectors after document edits.
181+
* **Distance**:
182+
+
183+
[source]
184+
----
185+
score = 1 - cosine(stored_vec, recomputed_vec)
186+
----
187+
+
188+
Same shape as Semantic but the comparands are vectors, not text.
189+
The implementation may skip recomputation if the document
190+
modality's `version` hasn't advanced past the vector's
191+
`vector_version_seen` watermark.
192+
* **Threshold default**: `0.10`.
193+
* **Example**: stored vector cosine-similarity 0.92 with recomputed
194+
vector → score `0.08` → at threshold.
195+
196+
## Consequences
197+
198+
### Positive
199+
200+
* Each detector implementation has an unambiguous specification.
201+
Two implementers ship the same numeric output for the same input.
202+
* The aggregator (`DriftReport::overall_score`) can take a weighted
203+
sum across categories without re-normalising — every category
204+
produces `[0, 1]`.
205+
* The CLI's `--threshold` flag has a well-defined meaning per
206+
category; users can override on a per-category basis without
207+
guessing what "drift score 0.2" means.
208+
* `verisimiser doctor` can verify the threshold defaults are
209+
configured consistently across deployments.
210+
211+
### Negative
212+
213+
* Semantic and Embedding require a choice of embedding model. The
214+
ADR leaves the model TBD; that choice is OQ-1 for the future.
215+
* Statistical's threshold for *unbounded* numeric fields requires
216+
a per-field scale constant. Deployments must configure this; we
217+
cannot pick a one-size-fits-all default.
218+
* Wasserstein-1 is more expensive than KL-divergence for large
219+
distributions. Detector implementations may need to subsample.
220+
221+
### Neutral
222+
223+
* Temporal stays unimplemented today. V-L1-E2 picks it up first
224+
precisely because the math is the cheapest.
225+
* The eight categories may later collapse — e.g. Embedding might
226+
fold into Semantic if the embedding model becomes the same. The
227+
ADR is the only place to record that choice.
228+
229+
## Open questions (track separately)
230+
231+
* **OQ-1**: Which sentence embedding model for Semantic and
232+
Embedding? Trade-off: model size vs. quality vs. licensing.
233+
Suggested follow-up ADR.
234+
* **OQ-2**: Per-field scale config for unbounded numeric fields in
235+
Statistical. Stored where in the manifest?
236+
* **OQ-3**: Drift report retention. Per V-L2-P1 the provenance log
237+
has a retention bound; drift reports might warrant their own.
238+
239+
## Cross-references
240+
241+
* V-L1-E2 (issue #49) implements the Temporal detector first.
242+
* V-L1-B1 / `docs/theory/provenance-threat-model.adoc` constrains
243+
what the Provenance drift category can claim under adversary.
244+
* `src/tier1/drift.rs::DriftCategory` is the canonical enum.

0 commit comments

Comments
 (0)