Skip to content

review-queue.md

Latest commit

 

History

History
108 lines (83 loc) · 4.78 KB

review-queue.md

File metadata and controls

108 lines (83 loc) · 4.78 KB
Raw

Human Review Queue

The review queue is the single human-facing surface that aggregates every item from this repository that a maintainer should look at before merging, publishing, or promoting anything.

Built by tools/build_review_queue.py. Outputs:

  • reports/review-queue.json — full machine-readable list.
  • reports/review-queue.md — human-readable summary.

What it is

A grouped list of items that fall into one of these categories:

Type Source What it represents
high-risk-workflow unified catalog workflows flagged high risk (e.g. webhook + code node combination)
repair-proposal reports/repair-proposals.json catalog-level repair proposals from the existing collection
runtime-repair-proposal reports/runtime-proof/<slug>/repair-proposals.json proposals emitted by the prompt-to-n8n proof loop
learning-event reports/runtime-proof/<slug>/learning-events.json evidence captured from validation findings
duplicate-candidate reports/duplicates-report.json exact-content or title-similar clusters flagged for dedupe
generated-pack-needs-behavioural-test manifest anchor item: the 420-template pack has not been imported into n8n
prompt-to-n8n-proof runtime-proof outputs one item per proof slug, asking for behavioural testing

Every item carries:

  • id — stable uuid5 derived from type + sourcePath + summary.
  • type — one of the above.
  • severity — low, medium, high, or critical.
  • sourcePath — path to the underlying file in the repo, relative to root.
  • summary — short human-readable description.
  • recommendedAction — what a reviewer should do.
  • status: "pending" — the queue builder never approves anything.
  • humanReviewRequired: true — always.

How to review repair proposals

Repair proposals conform to schemas/repair-proposal.schema.json.

For each proposal:

  1. Read diagnosis and proposedFix.
  2. Inspect the underlying workflow file at sourcePath.
  3. Decide one of:
    • accept — change the status field to accepted. A separate, explicit apply step is still required; nothing is auto-applied.
    • reject — change status to rejected and add a short note.
    • leave proposed — fine if more information is needed.
  4. Where possible, attach a validationAfter block by re-running the relevant validator with the patch applied locally.

Repair-proposal severity:

  • critical — credential leak, secret pattern, active-by-default generated workflow. Triage immediately.
  • high — missing trigger, duplicate node names, broken connection references, regulated-category workflow missing safety notes.
  • medium — webhook without authentication note, code node needing review, generated template missing setup notes.
  • low — missing or unknown category, status drift, cosmetic documentation gaps.

How to treat learning events

Learning events conform to schemas/learning-event.schema.json.

Each event records a class of validation finding and may include a proposedRule. None of these are applied automatically. Promotion of a learning event into the curated wiki or repair rules is a separate, human-driven decision.

For each event:

  1. Read failureType, failureSummary, and rootCause.
  2. If a proposedRule is present, decide whether the rule is correct, should be reshaped, or should be discarded.
  3. To promote a rule, open a PR that adds a hand-written entry under wiki/repair-rules/ or wiki/patterns/ (whichever is appropriate) and link to the learning event id in the PR description.
  4. Update the event's humanReviewStatus to approved or rejected and appliedToWiki to true only after the curated wiki PR lands.

No auto-approval, no auto-apply

This is a hard rule for this repository:

  • The review queue builder does not change any status.
  • Repair proposals are never applied by the tooling.
  • Learning events never write to curated wiki files.
  • The curated wiki sections (wiki/patterns/, wiki/integrations/, wiki/repair-rules/, wiki/failure-cases/, wiki/framework-guides/) are only modified by human PR.
  • The seed files under wiki/generated/ are regenerated by tools; they are not promoted into the curated wiki.

Pending items are not a failure signal

CI does not fail because the review queue has pending items. A non-zero queue is the expected, healthy state: it means the catalog and the runtime proof produced material that humans should look at.

The hard failure conditions are documented in maintainer-release-checklist.md: secret findings, invalid JSON, generated-pack validation failure, schema validation failure, and prompt-to-n8n tool crash.