Skip to content

Commit bb3d9f7

Browse files
montfortclaude
andcommitted
feat(baton): discontinue title-scan; declared work_verb is the sole signal (#332)
Implements decision #332 (registered after Sentinel calibration #331). The declared work_verb (+ design_provenance), captured at authoring, is now the sole authoritative classification signal. A unit with no declared verb is unclassifiable -> routed up to frontier conservatively + a nudge to declare the verb. Never guessed from the title. Why: the Sentinel calibration proved title-substring classification unsafe (high+medium precision 0.57, 4 errors *downward*) due to object-vs-verb collisions (a module named Audit), incidental tokens, and an unbounded per-adopter keyword treadmill (#321 anti-pattern). Fake savings are worse than an honest "declare the verb" gap. Removes signals::{Cue,scan_cues,CUE_TABLE,matches_at_word_start}. classify returns Option<TaskClass> (None = undeclared); implement + design_provenance=upstream degrades to operator (residual cognitive load). telemetry: undeclared_fraction replaces conflict_fraction; saving now provably rests on declared units. units harvests work_verb from charter frontmatter + follow-up lines. Suite rewritten (78 tests), clippy clean. Sentinel dogfood (read-only): the legacy corpus declares no verbs -> 100% undeclared -> frontier -> 0 saving + nudge. No fake savings. Framework graduation (first-class field + validator) deferred to a fw bump, gated on schema ratification. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent 6ad3cc3 commit bb3d9f7

11 files changed

Lines changed: 433 additions & 348 deletions

File tree

Lines changed: 101 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,101 @@
1+
---
2+
id: AILOG-2026-06-27-001
3+
title: Baton — discontinue title-scan; declared work_verb as the sole classification signal
4+
status: accepted
5+
created: 2026-06-27
6+
agent: claude-code-opus-4.8
7+
confidence: high
8+
review_required: true
9+
risk_level: low
10+
eu_ai_act_risk: not_applicable
11+
nist_genai_risks: []
12+
iso_42001_clause: []
13+
lines_changed: 470
14+
files_modified: [experiment-baton/src/signals.rs, experiment-baton/src/classify.rs, experiment-baton/src/route.rs, experiment-baton/src/telemetry.rs, experiment-baton/src/units.rs, experiment-baton/src/main.rs, experiment-baton/tests/dry_run_router.rs, experiment-baton/tests/fixtures/governance-corpus]
15+
observability_scope: none
16+
tags: [baton, phase2, classification, work-verb, title-scan, adopter-feedback]
17+
related: [AILOG-2026-06-26-002, CHARTER-03-dry-run-router]
18+
---
19+
20+
# AILOG: Baton — declared work_verb as the sole classification signal
21+
22+
## Summary
23+
24+
Implements decision **#332** (registered after the Sentinel adopter calibration,
25+
#331): **discontinue title-scan** as a classification mechanism. The declared
26+
**`work_verb`** (+ `design_provenance`), captured at authoring, becomes the sole
27+
authoritative classification signal. A unit with no declared verb is
28+
**unclassifiable** — routed up to frontier conservatively, with a nudge to declare
29+
the verb — never guessed from the title.
30+
31+
## Why (the adopter evidence)
32+
33+
The Sentinel calibration (#331, E1/E2/E3) proved title-substring classification is
34+
not just imprecise but **unsafe**: high+medium precision 0.57 with **4 errors
35+
*downward*** (frontier work routed cheap), all from reading the *object noun /
36+
incidental token* rather than the work verb — `interfaces/audit.go` (filename),
37+
`(commit hash)`, "Audit remediation", "make test-live". The root cause is
38+
structural, not tunable:
39+
40+
- **Object-vs-verb collisions are irreducible.** Sentinel has a module *named*
41+
`Audit`; "audit" as a domain noun (something you *implement*) is indistinguishable
42+
by keyword from "audit" as a verb (something you *do*).
43+
- **No schema to scan against.** The title is unconstrained, schema-less, often
44+
bilingual free text. Reinforcing the scan per adopter is an unbounded keyword
45+
treadmill — the #321 "calibrated to one stack" anti-pattern.
46+
- **Fake savings are worse than an honest gap.** Part of the scan's "saving" was
47+
false (critical work sent to a weak model). "Unknown → declare the verb" is the
48+
honest state.
49+
50+
## What changed
51+
52+
- **`signals.rs`** — removed the title-scan machinery (`Cue`, `scan_cues`,
53+
`CUE_TABLE`, `matches_at_word_start`). Added `WorkVerb`
54+
(design/implement/audit/operate) and `DesignProvenance` (new/upstream) with
55+
controlled-vocabulary parsing. `UnitSignals` carries the declared verb/provenance.
56+
- **`classify.rs`**`class: Option<TaskClass>`. Verb→class, with the
57+
residual-cognitive-load refinement: `implement` + `design_provenance=upstream`
58+
`operator` (instruments prior design = mechanical). Undeclared → `None`. A
59+
declared verb is High confidence (the author knew it for free).
60+
- **`route.rs`** — undeclared (`None`) → frontier (conservative default, R1); the
61+
high-risk Implementer escalation is retained.
62+
- **`telemetry.rs`**`conflict_fraction` (a title-scan artifact) replaced by
63+
`undeclared_fraction` (the actionable nudge metric). The saving now provably
64+
rests on declared units (`low_confidence_savings_fraction` → 0).
65+
- **`units.rs`** — harvest `work_verb`/`design_provenance` from charter frontmatter
66+
(`read_frontmatter_yaml`) and from follow-up `- **Work verb**:` /
67+
`- **Design provenance**:` lines. Batch/Task have no declaration slot in the
68+
prototype → undeclared.
69+
- **`main.rs`**`classify`/`route` rendering for `Option<class>` + the undeclared
70+
nudge.
71+
72+
## Verification
73+
74+
- `cargo test -p straymark-baton` ✓ — full suite rewritten for the new model
75+
(declared-verb mapping, upstream→operator degrade, undeclared→frontier, telemetry
76+
undeclared metric); `cargo clippy` clean.
77+
- **Sentinel dogfood** (read-only, `git status` intact): the legacy corpus declares
78+
no verbs → **100% undeclared → all routed to frontier → 0 saving → "not routable",
79+
with the "declare a work_verb" nudge.** That is the truthful clean-cut behaviour:
80+
no fake savings; the gap is an action, not a number.
81+
82+
## Scope boundary
83+
84+
This is the **Baton-side prototype** (self-contained in the experiment), de-risking
85+
before graduation. Deferred, on purpose:
86+
87+
- **Framework graduation**`work_verb`/`design_provenance` as first-class
88+
governance fields (schemas, templates, validator nudge, the authoring-time
89+
discipline) is a `fw-X.Y.Z` change, gated on schema ratification (the adopter
90+
explicitly asked not to instrument against an unratified schema).
91+
- **Finer-grained declaration** — how batch/task units declare a verb (no
92+
frontmatter slot yet) is part of the framework design.
93+
- **Forward-validation** — do real-world authors declare verbs correctly? — is
94+
StrayMark's job post-adoption over a varied corpus (#332).
95+
- **B2 auto-assist** — using the intent-provenance edges to *suggest*
96+
`design_provenance: upstream` when a unit only touches already-defined contracts.
97+
98+
## EU AI Act Considerations
99+
100+
Not applicable — local developer tooling; no automated decision-making, no personal
101+
data, no model inference. Read-only over the target tree (NFR1).

experiment-baton/specs/003-dry-run-router/spec.md

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -109,6 +109,13 @@ straymark-baton route [PATH] --dry-run [--out FMT] [--granularity G] [--confi
109109

110110
## 5. Classification rules (cheap-first, deterministic)
111111

112+
> **Superseded by #332 (2026-06-27).** The title-scan cue mechanism below was
113+
> discontinued after the Sentinel calibration (#331) proved it unsafe
114+
> (object-vs-verb collisions, per-adopter keyword treadmill). The authoritative
115+
> signal is now the **declared `work_verb`** (+ `design_provenance`); an undeclared
116+
> unit is unclassifiable → routed up + nudge. See `AILOG-2026-06-27-001`. The text
117+
> below is retained as the historical Phase-2 design.
118+
112119
Rules over precomputed signals; **no LLM** (NFR3). Calibrated on the adopter corpus (Sentinel now, N=2 later). Conservative: when signals are ambiguous or conflicting, **route up** (toward the more capable tier) — never down (NFR4 / R1).
113120

114121
| Signal pattern | Class |

experiment-baton/src/classify.rs

Lines changed: 65 additions & 139 deletions
Original file line numberDiff line numberDiff line change
@@ -1,21 +1,19 @@
1-
//! Cheap task classifier (Baton Phase 2, B3).
1+
//! Task classifier (Baton Phase 2, B3 — revised for #332).
22
//!
3-
//! Pure `classify(&UnitSignals) -> Classification` — deterministic rules over the
4-
//! cheap signals from B2, no LLM, no I/O (concept §4.2 corollary; NFR2/NFR3). Maps
5-
//! a unit to a `TaskClass` (Planner / Implementer / Auditor / Operator — the
6-
//! routing targets of §4.2) that B4 turns into a tier recommendation.
3+
//! Pure `classify(&UnitSignals) -> Classification`. The class comes from the
4+
//! **declared `work_verb`** (controlled vocabulary), with one refinement: an
5+
//! `implement` unit whose `design_provenance` is `upstream` only instruments
6+
//! prior design → it is mechanical → `operator` (the residual-cognitive-load
7+
//! principle from the Sentinel calibration, #331/#332).
78
//!
8-
//! Conservative bias (R1): when cues conflict, **route up** to the highest-tier
9-
//! class present; when nothing is known, default to the *escalatable middle*
10-
//! (Implementer) rather than claiming a unit is cheap (Operator). A false "this is
11-
//! mechanical" routes real work *down* and costs quality — the one error class we
12-
//! refuse. The `rationale` records the driving signals so a recommendation is
13-
//! auditable and calibration deltas are legible.
9+
//! A unit with **no declared verb is unclassifiable** (`class = None`): the
10+
//! router sends it up conservatively (frontier) and the telemetry nudges the
11+
//! author to declare the verb. We never guess a class from the title.
1412
1513
use serde::Serialize;
1614

1715
use crate::intent::Confidence;
18-
use crate::signals::{Cue, RiskLevel, UnitSignals};
16+
use crate::signals::{DesignProvenance, UnitSignals, WorkVerb};
1917

2018
/// The routing target — what kind of model tier this work wants (§4.2).
2119
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize)]
@@ -36,174 +34,102 @@ impl TaskClass {
3634
TaskClass::Operator => "operator",
3735
}
3836
}
39-
40-
/// Route-up rank: higher wins a conflict (Planner is the most capable tier;
41-
/// Implementer outranks Auditor because it can escalate to frontier).
42-
fn rank(self) -> u8 {
43-
match self {
44-
TaskClass::Planner => 4,
45-
TaskClass::Implementer => 3,
46-
TaskClass::Auditor => 2,
47-
TaskClass::Operator => 1,
48-
}
49-
}
5037
}
5138

52-
/// A classification with its confidence and the signals that drove it.
39+
/// A classification. `class = None` means the unit declared no work verb and is
40+
/// unclassifiable — route up + nudge, never guess.
5341
#[derive(Debug, Clone, Serialize)]
5442
pub struct Classification {
55-
pub class: TaskClass,
43+
pub class: Option<TaskClass>,
5644
pub confidence: Confidence,
57-
/// True when the unit's signals pointed at more than one class and we routed
58-
/// up — a within-unit heterogeneity marker the telemetry aggregates (§10.4).
59-
pub conflict: bool,
6045
pub rationale: String,
6146
}
6247

63-
/// The class a single cue points at.
64-
fn cue_class(cue: Cue) -> TaskClass {
65-
match cue {
66-
Cue::Architecture => TaskClass::Planner,
67-
Cue::Audit => TaskClass::Auditor,
68-
Cue::Implement | Cue::Fix => TaskClass::Implementer,
69-
Cue::Operate | Cue::Test => TaskClass::Operator,
70-
}
71-
}
72-
73-
/// Classify a unit from its cheap signals.
48+
/// Classify a unit from its declared signals.
7449
pub fn classify(s: &UnitSignals) -> Classification {
75-
let mut candidates: Vec<(TaskClass, String)> = s
76-
.cues
77-
.iter()
78-
.map(|&c| (cue_class(c), format!("{:?} cue", c)))
79-
.collect();
80-
81-
// High risk on an unbounded surface looks like open-ended design (§5).
82-
if s.risk_level == Some(RiskLevel::High) && s.surface_size == 0 {
83-
candidates.push((TaskClass::Planner, "high risk, unbounded surface".into()));
84-
}
85-
86-
if candidates.is_empty() {
87-
// No cue, no risk signal. Don't claim it's cheap — default to the
88-
// escalatable middle (Implementer), which B4 still lifts to frontier on
89-
// an escalation signal. Low confidence.
50+
let Some(verb) = s.work_verb else {
9051
return Classification {
91-
class: TaskClass::Implementer,
52+
class: None,
9253
confidence: Confidence::Low,
93-
conflict: false,
94-
rationale: "no cue or risk signal — default Implementer (route up from Operator)".into(),
54+
rationale: "no work_verb declared — unclassifiable; route up and declare the verb".into(),
9555
};
96-
}
97-
98-
// Route up: the highest-rank class present wins any conflict.
99-
let class = candidates
100-
.iter()
101-
.map(|(c, _)| *c)
102-
.max_by_key(|c| c.rank())
103-
.unwrap();
104-
let conflicting = candidates.iter().any(|(c, _)| c.rank() != class.rank());
105-
106-
let confidence = if conflicting {
107-
Confidence::Low
108-
} else if corroborated(class, s) {
109-
Confidence::High
110-
} else {
111-
Confidence::Medium
11256
};
11357

114-
let mut reasons: Vec<String> = candidates
115-
.iter()
116-
.filter(|(c, _)| *c == class)
117-
.map(|(_, r)| r.clone())
118-
.collect();
119-
if conflicting {
120-
let dropped: Vec<&str> = candidates
121-
.iter()
122-
.filter(|(c, _)| *c != class)
123-
.map(|(c, _)| c.as_str())
124-
.collect();
125-
reasons.push(format!("routed up over {}", dropped.join(", ")));
58+
// `implement` that only instruments upstream design is mechanical → operator.
59+
if verb == WorkVerb::Implement && s.design_provenance == Some(DesignProvenance::Upstream) {
60+
return Classification {
61+
class: Some(TaskClass::Operator),
62+
confidence: Confidence::High,
63+
rationale: "work_verb=implement + design_provenance=upstream → operator (instruments prior design)".into(),
64+
};
12665
}
12766

67+
let class = match verb {
68+
WorkVerb::Design => TaskClass::Planner,
69+
WorkVerb::Implement => TaskClass::Implementer,
70+
WorkVerb::Audit => TaskClass::Auditor,
71+
WorkVerb::Operate => TaskClass::Operator,
72+
};
12873
Classification {
129-
class,
130-
confidence,
131-
conflict: conflicting,
132-
rationale: reasons.join("; "),
74+
// A declared verb is authoritative — the author knew it for free.
75+
class: Some(class),
76+
confidence: Confidence::High,
77+
rationale: format!("work_verb={}", verb_str(verb)),
13378
}
13479
}
13580

136-
/// A non-cue signal that corroborates the class → High confidence.
137-
fn corroborated(class: TaskClass, s: &UnitSignals) -> bool {
138-
let effort = s.effort_estimate.as_deref();
139-
match class {
140-
TaskClass::Operator => effort == Some("XS"),
141-
TaskClass::Implementer => matches!(effort, Some("XS" | "S" | "M")),
142-
TaskClass::Planner => effort == Some("L") || s.risk_level == Some(RiskLevel::High),
143-
// An audit cue alone is enough to act on, but nothing else corroborates it.
144-
TaskClass::Auditor => false,
81+
fn verb_str(v: WorkVerb) -> &'static str {
82+
match v {
83+
WorkVerb::Design => "design",
84+
WorkVerb::Implement => "implement",
85+
WorkVerb::Audit => "audit",
86+
WorkVerb::Operate => "operate",
14587
}
14688
}
14789

14890
#[cfg(test)]
14991
mod tests {
15092
use super::*;
15193

152-
fn sig(cues: &[Cue]) -> UnitSignals {
94+
fn sig(verb: Option<WorkVerb>, prov: Option<DesignProvenance>) -> UnitSignals {
15395
UnitSignals {
154-
cues: cues.to_vec(),
96+
work_verb: verb,
97+
design_provenance: prov,
15598
..Default::default()
15699
}
157100
}
158101

159102
#[test]
160-
fn each_cue_maps_to_its_class() {
161-
assert_eq!(classify(&sig(&[Cue::Architecture])).class, TaskClass::Planner);
162-
assert_eq!(classify(&sig(&[Cue::Audit])).class, TaskClass::Auditor);
163-
assert_eq!(classify(&sig(&[Cue::Implement])).class, TaskClass::Implementer);
164-
assert_eq!(classify(&sig(&[Cue::Fix])).class, TaskClass::Implementer);
165-
assert_eq!(classify(&sig(&[Cue::Operate])).class, TaskClass::Operator);
166-
assert_eq!(classify(&sig(&[Cue::Test])).class, TaskClass::Operator);
167-
}
168-
169-
#[test]
170-
fn conflict_routes_up_with_low_confidence() {
171-
// A title that is both architecture and cleanup → Planner (route up).
172-
let c = classify(&sig(&[Cue::Architecture, Cue::Operate]));
173-
assert_eq!(c.class, TaskClass::Planner);
174-
assert_eq!(c.confidence, Confidence::Low);
175-
assert!(c.rationale.contains("routed up over"));
103+
fn declared_verb_maps_to_class_at_high_confidence() {
104+
for (v, c) in [
105+
(WorkVerb::Design, TaskClass::Planner),
106+
(WorkVerb::Implement, TaskClass::Implementer),
107+
(WorkVerb::Audit, TaskClass::Auditor),
108+
(WorkVerb::Operate, TaskClass::Operator),
109+
] {
110+
let r = classify(&sig(Some(v), None));
111+
assert_eq!(r.class, Some(c));
112+
assert_eq!(r.confidence, Confidence::High);
113+
}
176114
}
177115

178116
#[test]
179-
fn no_signal_defaults_to_escalatable_middle() {
180-
let c = classify(&UnitSignals::default());
181-
assert_eq!(c.class, TaskClass::Implementer);
182-
assert_eq!(c.confidence, Confidence::Low);
183-
assert!(c.rationale.contains("no cue"));
117+
fn implement_upstream_degrades_to_operator() {
118+
let r = classify(&sig(Some(WorkVerb::Implement), Some(DesignProvenance::Upstream)));
119+
assert_eq!(r.class, Some(TaskClass::Operator));
120+
assert!(r.rationale.contains("upstream"));
184121
}
185122

186123
#[test]
187-
fn high_risk_unbounded_surface_is_planner() {
188-
let s = UnitSignals {
189-
risk_level: Some(RiskLevel::High),
190-
surface_size: 0,
191-
..Default::default()
192-
};
193-
assert_eq!(classify(&s).class, TaskClass::Planner);
124+
fn implement_new_stays_implementer() {
125+
let r = classify(&sig(Some(WorkVerb::Implement), Some(DesignProvenance::New)));
126+
assert_eq!(r.class, Some(TaskClass::Implementer));
194127
}
195128

196129
#[test]
197-
fn corroborating_effort_lifts_confidence_to_high() {
198-
let mut s = sig(&[Cue::Operate]);
199-
s.effort_estimate = Some("XS".into());
200-
assert_eq!(classify(&s).confidence, Confidence::High);
201-
202-
let mut s = sig(&[Cue::Implement]);
203-
s.effort_estimate = Some("S".into());
204-
assert_eq!(classify(&s).confidence, Confidence::High);
205-
206-
// Cue alone, no corroboration → Medium.
207-
assert_eq!(classify(&sig(&[Cue::Implement])).confidence, Confidence::Medium);
130+
fn undeclared_is_unclassifiable() {
131+
let r = classify(&sig(None, None));
132+
assert_eq!(r.class, None);
133+
assert!(r.rationale.contains("no work_verb"));
208134
}
209135
}

0 commit comments

Comments
 (0)