Skip to content

Commit 56e0939

Browse files
committed
feat: add data flywheel trainer artifacts
1 parent 5d8bc82 commit 56e0939

18 files changed

Lines changed: 1061 additions & 1 deletion

.agent/AGENTS.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -55,6 +55,9 @@ Daily driver, highest-leverage first:
5555
in one shot (stage + graduate + render). For rules you already know.
5656
- `show.py` — one-screen dashboard of brain state: episodes, candidates,
5757
lessons, failing skills, activity graph.
58+
- `data_flywheel_export.py` — local export of approved, redacted runs into
59+
trace records, context cards, eval cases, training-ready JSONL, and
60+
flywheel metrics. It does not train models or call APIs.
5861
- `list_candidates.py` / `graduate.py` / `reject.py` / `reopen.py` — review
5962
protocol for patterns the dream cycle has staged.
6063
- `memory_reflect.py <skill> <action> <outcome>` — log a significant event.

.agent/skills/_index.md

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -26,3 +26,11 @@ Pre-deployment verification against a structured checklist.
2626
Triggers: "deploy", "ship", "release", "go live"
2727
Constraints: all tests passing, no unresolved TODOs in diff,
2828
requires human approval for production.
29+
30+
## data-flywheel
31+
Turns approved, redacted runs into reusable local artifacts: trace records,
32+
context cards, eval cases, training-ready JSONL, and flywheel metrics.
33+
Triggers: "data flywheel", "trace to train", "training traces",
34+
"context cards", "eval cases", "approved runs", "vertical intelligence"
35+
Constraints: local-only by default; human-approved runs only; redaction required
36+
before trainable; does not train models.

.agent/skills/_manifest.jsonl

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,3 +3,4 @@
33
{"name":"git-proxy","version":"2026-01-01","triggers":["commit","push","branch","merge","rebase","pull request","PR"],"tools":["bash"],"preconditions":[".git exists"],"constraints":["never force push to main","never force push to protected branches","run tests before push"],"category":"operations"}
44
{"name":"debug-investigator","version":"2026-01-01","triggers":["debug","why is this failing","investigate","stack trace","bug"],"tools":["bash","memory_reflect"],"preconditions":[],"constraints":["reproduce before fixing","fix root cause, not symptoms"],"category":"engineering"}
55
{"name":"deploy-checklist","version":"2026-01-01","triggers":["deploy","ship","release","go live"],"tools":["bash"],"preconditions":[],"constraints":["all tests passing","no unresolved TODOs in diff","requires human approval for production"],"category":"operations"}
6+
{"name":"data-flywheel","version":"2026-04-25","triggers":["data flywheel","trace to train","training traces","context cards","eval cases","approved runs","vertical intelligence"],"tools":["bash","git"],"preconditions":[".agent exists"],"constraints":["local-only by default","human-approved runs only","redaction required before trainable","do not train models"],"category":"operations"}
Lines changed: 112 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,112 @@
1+
---
2+
name: data-flywheel
3+
version: 2026-04-25
4+
triggers: ["data flywheel", "trace to train", "training traces", "context cards", "eval cases", "approved runs", "vertical intelligence"]
5+
tools: [bash, git]
6+
preconditions: [".agent exists"]
7+
constraints: ["local-only by default", "human-approved runs only", "redaction required before trainable", "do not train models"]
8+
---
9+
10+
# Data Flywheel - approved runs into reusable intelligence
11+
12+
Use this skill when a user wants to turn repeated human-approved agent work
13+
across Claude Code, Hermes, OpenClaw, Codex, Cursor, or custom `.agent/` loops
14+
into local artifacts for retrieval, evals, prompt shrinking, and optional
15+
future open-weight model/adapters.
16+
17+
The flywheel is:
18+
19+
```text
20+
approved run
21+
-> redacted trace
22+
-> context card
23+
-> eval case
24+
-> training-ready JSONL
25+
-> optional downstream SLM/adapter experiment later
26+
```
27+
28+
This skill creates the harness. It does not train a model.
29+
30+
## Hard Rules
31+
32+
- Use only human-approved runs. Rejected or unknown-review runs can become
33+
failure-mode notes, not trainable examples.
34+
- Redaction must pass before anything is marked trainable.
35+
- Do not store raw prompts, raw code, client names, addresses, phone numbers,
36+
emails, secrets, credentials, or unredacted CRM records.
37+
- Keep `.agent/flywheel/` private and gitignored unless the user explicitly
38+
commits sanitized examples.
39+
- Stay model-agnostic. Mention model families only as downstream examples.
40+
41+
## Inputs
42+
43+
Default local input:
44+
45+
```text
46+
.agent/flywheel/approved-runs.jsonl
47+
```
48+
49+
Each line should be a sanitized run record with:
50+
51+
- `domain`
52+
- `workflow`
53+
- `harness`
54+
- `instruction`
55+
- `input_redacted`
56+
- `output_approved`
57+
- `human_review.status` as `accepted` or `edited`
58+
- `redaction_status: passed`
59+
- `pii_level`
60+
- optional `stable_rules`, `tool_contracts`, `eval_tags`, `failure_modes`
61+
62+
## Export
63+
64+
Run:
65+
66+
```bash
67+
python3 .agent/tools/data_flywheel_export.py
68+
```
69+
70+
Outputs go to:
71+
72+
```text
73+
.agent/flywheel/exports/<YYYY-MM-DD>/
74+
```
75+
76+
Key outputs:
77+
78+
- `trace-records.jsonl`
79+
- `training-examples.jsonl`
80+
- `eval-cases.jsonl`
81+
- `context-cards/<domain>/<workflow>.md`
82+
- `context-cards/<domain>/<workflow>.json`
83+
- `flywheel-metrics.json`
84+
85+
## Readiness Checks
86+
87+
Use these as heuristics, not hard rules:
88+
89+
- 10-25 approved runs: useful first context card
90+
- 25-100 approved runs: first eval set and repeated failure modes
91+
- 100-300 approved runs: context compression and routing measurement
92+
- 500-1,500 high-quality examples: narrow adapter experiment candidate
93+
- 2,000-10,000+ examples: broader workflow-family corpus
94+
95+
## What To Report
96+
97+
When finishing, report:
98+
99+
- traces exported
100+
- trainable examples exported
101+
- eval cases exported
102+
- context cards created
103+
- redaction pass rate
104+
- acceptance rate by workflow
105+
- workflows that should stay frontier-model/manual-review
106+
- workflows that may become SLM/adapter candidates later
107+
108+
## Self-rewrite hook
109+
110+
If users repeatedly ask for the same domain-specific fields, add them to a
111+
local context card or schema example instead of hard-coding them into this
112+
general skill.

0 commit comments

Comments
 (0)