Skip to content

Commit e110ec2

Browse files
Roadmap: evidence-gathering methodology (playbooks, ingest adapters, doctor, record proxy)
1 parent 8381d1d commit e110ec2

1 file changed

Lines changed: 30 additions & 0 deletions

File tree

ROADMAP.md

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -74,6 +74,27 @@ Catalog work (the content is the product):
7474
- **NIST AI RMF catalog** (GOVERN / MAP / MEASURE / MANAGE) — the
7575
planned fast-follow from v0.1.
7676

77+
Getting the evidence in the first place (the biggest v0.1 gap —
78+
most teams don't have governance-grade logs lying around, and
79+
telling them "feed us JSONL" isn't a methodology):
80+
81+
- **Evidence playbooks** — per-platform guides in `docs/evidence/`
82+
for exporting usable logs from the systems people actually run:
83+
OpenAI (stored completions / usage export), Azure OpenAI
84+
(diagnostic settings → Log Analytics, query included), AWS Bedrock
85+
(model-invocation logging → S3), LiteLLM (file callback), LangSmith
86+
run exports, plus a "roll your own" spec listing the minimal fields
87+
Compass needs. Copy-paste commands, not prose.
88+
- **Named ingest adapters** — `compass ingest --from
89+
openai|litellm|bedrock|csv` to normalise native export formats into
90+
Compass JSONL. The CSV adapter doubles as the approvals bridge:
91+
teams tracking approvals in Jira or ServiceNow export a CSV and the
92+
human-oversight check lights up.
93+
- **`compass doctor`** — an evidence gap report: looks at what you've
94+
provided, lists which checks can't run, and prints the specific
95+
playbook steps to close each gap. If you can't produce a record,
96+
that absence is itself a finding.
97+
7798
Evidence + CI:
7899

79100
- **MCP trust-registry check** — ingest an exported server allowlist
@@ -92,6 +113,15 @@ Evidence + CI:
92113

93114
## v0.3 — crosswalks, trends, and richer evidence
94115

116+
- **`compass record`** — solve the cold-start problem for teams with
117+
no logs at all: a localhost pass-through proxy speaking the
118+
OpenAI-compatible API. Point your SDK's `base_url` at it and every
119+
request and tool call is written as Compass-ready JSONL,
120+
hash-chained as it's written so self-collected evidence is
121+
tamper-evident too. Local only, no accounts, nothing leaves the
122+
machine. Start recording today, assess in two weeks.
123+
- **Approval-system adapters** — Jira and ServiceNow export
124+
converters building on the v0.2 CSV bridge.
95125
- **Framework crosswalk** — map equivalent controls across EU / IMDA /
96126
NIST so one answered assessment scores against every catalog
97127
("answer once, score everywhere"), with per-framework overrides.

0 commit comments

Comments
 (0)