Skip to content

pipeline-model.md

Latest commit

 

History

History
40 lines (26 loc) · 3.21 KB

pipeline-model.md

File metadata and controls

40 lines (26 loc) · 3.21 KB
Raw

Pipeline model

agent-skill-forge runs a fixed, skill-specific pipeline. It is deliberately not a general-purpose CI/CD engine: the stage list is known, the artifacts are typed, and the only dynamic control flow is the repair loop.

Stages

# Stage Input Output artifacts
1 parse_requirement skill requirement YAML input/skill-request.yaml, artifacts/normalized-requirement.json
2 generate_spec normalized requirement artifacts/skill-spec.json
3 generate_skill_package skill spec workspace/generated-skill/ (SKILL.md, manifest, tools.json, examples, README)
4 generate_eval_cases skill spec workspace/generated-skill/eval-cases.json, verification-rules.json
5 run_verification skill package artifacts/verification-summary.json, verification-events.jsonl, reports/verification-report.html, replay-artifacts/*.json
6 analyze_failures verification result artifacts/failure-analysis.json
7 repair_if_failed package + failure analysis artifacts/repair-plan-attempt-<n>.json, workspace/repaired-skill-attempt-<n>/
8 rerun_verification repaired package updated verification artifacts, before/after metric comparison
9 package_if_passed final package + gate result dist/<name>-<version>.zip (run dir and project dist/)

Stages 1–5 are critical: if one fails (invalid YAML, stub adapter, invalid model output), the pipeline aborts, remaining stages are recorded as skipped, and reports are still written for debugging.

The repair loop

After analyze_failures, while the quality gate fails and repair attempts remain (default max: 2):

  1. repair_if_failed — turn the failure analysis into a repair plan, hand the current package to the model adapter's repairSkill, and write the result to a new repaired-skill-attempt-<n> directory. A guard rejects any modification to eval-cases.json or verification-rules.json: repairs must improve the skill, never weaken its tests.
  2. rerun_verification — verify the repaired package, re-evaluate the gate, re-analyze failures, and record the before/after pass rate (improvement or regression).

When the gate passes on the first verification, both stages are recorded as skipped with an explanatory note.

Contracts between stages

Stages communicate through a typed, mutable PipelineState (see src/core/stage.ts) and persist every intermediate to disk. Model adapter outputs are re-validated with zod schemas at the stage boundary — an adapter that returns a malformed spec or eval case set fails the stage with a clear error instead of corrupting downstream stages.

Run directory

Each run gets a canonical runs/run-<timestamp>-<suffix>/ directory. At the end of the run (pass or fail) it is mirrored — by copy, for Windows portability — to the --output directory (default runs/latest). The final package zip is additionally copied to the project-level dist/ directory.

Determinism

With --model mock and --verifier local-demo, the pipeline is deterministic apart from timestamps and run IDs: same input YAML, same generated package, same metrics. This is what makes the demo and CI reliable.