Skip to content

Commit 54f81e0

Browse files
committed
docs: define enterprise-readiness test harness boundary
1 parent 03fd837 commit 54f81e0

1 file changed

Lines changed: 111 additions & 0 deletions

File tree

Lines changed: 111 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,111 @@
1+
# Enterprise-Readiness Test Harness
2+
3+
## Status
4+
5+
**Version:** v0.1
6+
**Scope:** Synthetic / mocked downstream systems / CI replay
7+
**Claim:** Test-harness evidence only
8+
9+
This document defines what the enterprise-shaped scenario harness currently proves and what remains absent.
10+
11+
## Clean claim
12+
13+
This harness tests whether a bounded AI-assisted action can reach a mocked downstream system when required authority is missing.
14+
15+
For ESP-001, the answer is:
16+
17+
```text
18+
missing authority_token
19+
-> DENY
20+
-> mock email adapter not called
21+
-> downstream_send=false
22+
-> receipt_written=true
23+
```
24+
25+
## What this harness proves
26+
27+
On the synthetic path, the harness proves:
28+
29+
- the action class is named: `SEND_EXTERNAL_EMAIL`
30+
- the required authority field is missing: `authority_token`
31+
- the gate returns `DENY`
32+
- the mocked downstream email adapter is not called
33+
- the adapter records zero sent messages
34+
- a receipt is generated
35+
- the scenario can be replayed by pytest
36+
- the test can run in GitHub Actions
37+
38+
## What this harness does not prove
39+
40+
This harness does not prove:
41+
42+
- live SMTP / API non-execution
43+
- production runtime enforcement
44+
- enterprise deployment
45+
- certification
46+
- compliance
47+
- legal adequacy
48+
- closure of all bypass paths
49+
- no dispatch through queues, retries, alternate APIs, human handoff, or connector side channels
50+
- real-world controlled application
51+
- external review
52+
53+
## Required evidence for stronger claims
54+
55+
| Stronger claim | Required evidence |
56+
|---|---|
57+
| Live runtime enforcement | Gate integrated with a real execution layer or realistic service boundary |
58+
| Downstream non-execution | External connector call logs or service-level event evidence |
59+
| Path-universal control | Architecture showing every consequence path routes through the gate or is explicitly out of scope |
60+
| Enterprise readiness | Multiple realistic enterprise workflows, CI replay, bypass tests, external review, and controlled organisational pilot evidence |
61+
| Production readiness | Deployment architecture, operational monitoring, failure handling, persistence, security review, and live audit trail |
62+
| Certification / compliance | Independent qualified assessment against a named framework or legal requirement |
63+
64+
## Current label
65+
66+
Safe:
67+
68+
```text
69+
enterprise-readiness test harness
70+
```
71+
72+
Unsafe:
73+
74+
```text
75+
enterprise-ready system
76+
enterprise deployment evidence
77+
production enforcement proof
78+
certified control
79+
compliance-ready system
80+
```
81+
82+
## Next hardening path
83+
84+
1. Add more mocked consequence classes:
85+
- payment instruction
86+
- access change
87+
- HR decision support
88+
- vendor risk escalation
89+
90+
2. Add bypass tests:
91+
- retry cannot dispatch after denial
92+
- queue cannot dispatch after denial
93+
- stale DecisionRecord rejected
94+
- alternate send path blocked
95+
- audit failure fails closed
96+
97+
3. Add evidence matrix:
98+
- claim
99+
- required evidence
100+
- current proof
101+
- missing proof
102+
- next test
103+
104+
4. Add external review route:
105+
- issue template for hostile review
106+
- reviewer checklist
107+
- reproducibility instructions
108+
109+
## Stop rule
110+
111+
Do not call this enterprise-ready until an external reviewer or bounded organisation can inspect, run, and challenge the harness against a realistic workflow.

0 commit comments

Comments
 (0)