feat: make workflow capabilities self-documenting#100
Conversation
3248d99 to
4c137b7
Compare
There was a problem hiding this comment.
Seriously impressive piece of work, @Whamp. Reviewed and tested it end to end — the runtime capability contract, #93 preservation, the skill + examples, and the new release gates, plus a real-pi smoke on 0.80.10 — and it's in excellent shape: #93's arm/authorize path is untouched, the tool schema stays a single top-level object (no repeat of the recent provider-400), the runtime globals are behavior-preserved, 1027 tests and release:verify are green, and the roughly 9.5KB → 3.8KB context win is real and measured. The evidence discipline — generated docs checked for drift, examples that actually run, provider calls kept out of CI — is exactly right. Thank you for the depth here.
One small fast-follow, non-blocking: there's no refresh path for the frozen-guidance hashes — editing SKILL.md (or a frozen reference/example) trips a hard PROTECTED_GUIDANCE_DRIFT with no *:generate to recompute the hardcoded SHA-256s, and the error doesn't hint how to fix it. Is that manual step deliberate — a conscious "you're touching protected guidance" gate, in which case a line in CONTRIBUTING would cover it — or would you want a small guidance:freeze script to match your other *:generate surfaces? Happy to send either your way.
Otherwise this is good to go — give me the nod and I'll merge.
|
Ok thank you and great feedback let me see what I can do to clean that up. |
Summary
workflow-authoringskill with focused references and 11 runtime-tested examples;Closes #65.
Implements the capability-contract design from Whamp/pi-dynamic-workflows#22. Adds evidence for the background-versus-inline decision discussed in #89 without changing #93's trigger authorization behavior.
Context impact
Compared with
origin/mainat818fdd8:Detailed authoring guidance now loads on demand. In the comparative nine-model coverage sample, the optimized skill loaded a median 2,564 tokens versus 8,096 for the pre-trim skill, a 68.3% reduction.
See
docs/workflow-prompt-guidance-rationale.mdfor the reviewed record of prompt insertions, removals, and compactions. Seedocs/workflow-authoring-evidence.mdfor methodology, model matrix, per-scenario results, and reproduction commands.Capability and quality evidence
resumeFromRunId.agentTypevalues remain outside static prompts and generated references.gate()recipe.Post-tuning provider-backed validation:
This is not an independent holdout: the same scenario families informed the guidance changes. An earlier corrected-harness sample scored pre-trim at 75/81 and optimized at 71/81, so provider scores remain sampled, non-blocking evidence. The final candidate did not repeat the earlier nine-model, 162-case core panel.
The delivery-choice harness has deterministic tests and an optional provider CLI; this PR reports no provider-run delivery-choice result. Provider calls are not part of normal tests, release checks, or publishing.
Compatibility
promptGuidelinesgate and default background delivery;gate().okcompatibility path;consoleavailable for saved workflows while teachinglog();agentTypeconfiguration format;Verification
npm run checknpm run buildnpm test— 1,027/1,027 passednpm run docs:checknpm run context:checknpm run guidance:checknpm run release:verify— 0 warningsnpm pack --dry-run— 166 files; all 27 workflow-authoring skill files includedgit diff --checkThe PR is one atomic commit based on current
origin/main. Raw.autoexperiment data is intentionally excluded from both the commit and package.