Skip to content

intelligence-core-v1.md

Latest commit

 

History

History
239 lines (193 loc) · 9.82 KB

intelligence-core-v1.md

File metadata and controls

239 lines (193 loc) · 9.82 KB
Raw

Intelligence Core V1 — Implementation Report

Generated: 2026-05-24

Summary

Added a small intelligence/tooling layer on top of the existing workflow collection and the V0 expansion pack. The repository now has: a unified catalog with a deterministic quality score, a search CLI, a schema validator with a lightweight-fallback mode, a duplicate analyzer, a human-reviewable repair-proposal generator, a deterministic prompt-to-IR MVP, and a wiki-seed builder. CI gates run audit + validate + unify + schema-validate on every push.

No workflows were modified. No repair was applied. The expansion pack remains structurally valid (420/420) and the audit reports 0 possible secret findings across 3,613 JSON files.

What was implemented

  • tools/build_unified_catalog.py — merges main + generated catalogs into catalog/unified-workflows.index.json with deterministic qualityScore, qualityFlags, reviewStatus, and origin.
  • tools/search_workflows.py — keyword + filter CLI (--query, --category, --integration, --framework, --risk, --origin, --trigger, --review, --top, --json).
  • tools/validate_schemas.py — validates schemas + catalogs + sampled IR files. Uses jsonschema if available; lightweight required-field
    • enum check otherwise. Honestly reports which mode it ran in.
  • tools/analyze_duplicates.py — combines audit-hash duplicates with title-similarity clusters; per-cluster recommendation is one of keep / review / candidate-for-dedupe. No file modification.
  • tools/propose_repairs.py — emits proposals against schemas/repair-proposal.schema.json. Every proposal carries requiresHumanReview: true and status: "proposed". Capped at 500.
  • tools/prompt_to_ir.py — deterministic rule-based draft IR builder. Not an LLM. Outputs *.workflow.ir.json + *.md per prompt.
  • tools/build_wiki_seed.py — writes machine-generated seed notes under wiki/generated/. Does not touch the curated wiki.
  • .github/workflows/audit.yml — CI gate set: audit, generated-pack validation, unified-catalog build, schema validation. Fails on possibleSecretFindings > 0 or invalidJsonFiles > 0.
  • docs/ci-and-quality-gates.md — documents the gate set.
  • schemas/workflow-ir.schema.json — additive: added "generated" to the frameworkSource enum (required so the V0 IRs validate).
  • README.md — added "Intelligence Core V1" section, honestly scoped.

Files created / modified

Created

  • tools/build_unified_catalog.py
  • tools/search_workflows.py
  • tools/validate_schemas.py
  • tools/analyze_duplicates.py
  • tools/propose_repairs.py
  • tools/prompt_to_ir.py
  • tools/build_wiki_seed.py
  • .github/workflows/audit.yml
  • docs/ci-and-quality-gates.md
  • catalog/unified-workflows.index.json
  • reports/unified-catalog-report.{json,md}
  • reports/schema-validation.{json,md}
  • reports/duplicates-report.{json,md}
  • reports/repair-proposals.{json,md}
  • reports/prompt-to-ir-samples/create-a-workflow-that-receives-website-leads-sc.{workflow.ir.json,md}
  • reports/intelligence-core-v1.{json,md}
  • wiki/generated/{README,category-summary,integration-summary,trigger-patterns,risk-review-queue,prompt-patterns}.md

Modified (additive)

  • schemas/workflow-ir.schema.json (enum extension)
  • README.md (new Intelligence Core V1 section)

Refreshed by audit (existing behaviour)

  • catalog/workflows.index.json
  • reports/workflow-audit.{json,md}

No existing workflow file was deleted or overwritten.

Commands run

python tools/audit_workflows.py
python tools/validate_generated_pack.py
python tools/build_unified_catalog.py
python tools/validate_schemas.py
python tools/analyze_duplicates.py
python tools/propose_repairs.py
python tools/build_wiki_seed.py
python tools/search_workflows.py --query "lead qualification" --top 5
python tools/prompt_to_ir.py "Create a workflow that receives website leads, scores them, saves them to CRM, and alerts Slack"

All commands exited 0.

Validation results

Gate Result
audit: possibleSecretFindings 0
audit: invalidJsonFiles 0
generated pack 420/420 valid, no dup IDs/slugs, no structural failures
unified catalog build 3181 entries (420 generated + 2761 existing)
schema validation schemasParse / catalogsParse / catalogEntriesValid / generatedIrValid — all PASS (lightweight mode)
duplicates no duplicate IDs/slugs in generated pack
repair proposals 500 emitted, none auto-applied

Unified catalog stats

  • Total entries: 3,181
  • Origins: generated-pack=420, existing-library=2761
  • Risk: low=1637, medium=1311, high=233
  • Review status: review-suggested=381, ok-for-import=2800
  • Quality score: min=64, max=92, avg=86.05; 381 entries below 80, 0 below 60

Search CLI sample result

$ python tools/search_workflows.py --query "lead qualification" --top 5
Found 1 match(es). Showing top 1.

#1  score=12  quality=70  review=review-suggested
     title       : Lead Qualification with BatchData
     id          : 1910_Code_Webhook_Automation_Webhook-4da7422289
     origin      : existing-library
     framework   : n8n
     category    : automation
     trigger     : webhook
     integrations: code, httpRequest, if, slack, stickyNote, webhook
     riskLevel   : high
     sourcePath  : automation/1910_Code_Webhook_Automation_Webhook.json
     flags       : missing-description, risk-level-high, inline-code-node, webhook-trigger

Prompt-to-IR sample result

Prompt: "Create a workflow that receives website leads, scores them, saves them to CRM, and alerts Slack"

  • Inferred category: sales
  • Inferred trigger : webhook
  • Inferred integrations: slack, crm
  • Steps: 4 (trigger → score (code) → save (http) → alert (http))
  • Output: reports/prompt-to-ir-samples/create-a-workflow-that-receives-website-leads-sc.workflow.ir.json (+ .md)
  • IR generationStatus: "planned", validationStatus: "needs-review" — explicitly a draft.

Repair proposal count

  • Total: 500 (cap)
  • By severity: high=233, medium=267
  • By detected issue: high-risk-workflow=233, inline-code-node-review=223, webhook-needs-safety-note=44
  • All proposals carry requiresHumanReview: true and status: "proposed".

Wiki seed outputs

wiki/generated/:

  • category-summary.md
  • integration-summary.md
  • trigger-patterns.md
  • risk-review-queue.md
  • prompt-patterns.md
  • README.md

All files are tagged "Generated seed notes for human review. Not automatically promoted rules." The curated wiki sections under wiki/patterns/, wiki/integrations/, wiki/repair-rules/, wiki/failure-cases/, wiki/framework-guides/ are unchanged.

CI file created

.github/workflows/audit.yml — runs audit → validate-generated-pack → build-unified-catalog → validate-schemas on push, PR, and manual dispatch. Explicit failure conditions: possibleSecretFindings > 0 or invalidJsonFiles > 0. Uploads reports as artefacts on every run.

What is still not implemented

  • Behavioural validation. No generated workflow has been imported into n8n or executed.
  • Multi-framework export. Only n8n. Dify / LangGraph / Make / Zapier remain documented and planned.
  • LLM-backed prompt-to-workflow generator. The current tool is a deterministic rule-based MVP.
  • Auto-apply of repair proposals. Intentionally never; proposals are emitted to reports/ and stop there.
  • LLM Wiki learning loop. wiki/generated/ is seed material; the curated wiki entries are not generated.
  • Full JSON Schema validation in CI. CI runs in lightweight mode because jsonschema is not installed; documented as a limitation.

Risks / blockers

  • The repair-proposal queue is capped at 500 to stay reviewable; the underlying signal (233 high-risk + 223 code-node workflows) suggests the full review backlog is larger.
  • The 289 exact-content duplicate groups from the pre-existing collection are unchanged. The duplicate analyzer flags them; deduping is a separate human pass.
  • Title-similarity has a 275-cluster long tail; many are intentional (per-category templates in the generated pack are marked keep) but reviewers should still skim the cross-origin clusters.

Ready for git migration?

Yes — for review only. No commits, no pushes, no branches were created. The diff set is:

  1. schemas/workflow-ir.schema.json — one enum extension.
  2. tools/*.py — seven new tools (no modifications to existing tools).
  3. .github/workflows/audit.yml — new CI workflow.
  4. docs/ci-and-quality-gates.md — new docs page.
  5. README.md — additive Intelligence Core V1 section.
  6. catalog/unified-workflows.index.json — new generated artefact.
  7. reports/* — new generated reports.
  8. wiki/generated/ — new generated folder; existing wiki sections untouched.
  9. Audit refreshed catalog/workflows.index.json and reports/workflow-audit.{json,md} (existing behaviour).

Recommended commit strategy

  1. schema-enum — schemas/workflow-ir.schema.json (single-line enum addition).
  2. tools — tools/build_unified_catalog.py, tools/search_workflows.py, tools/validate_schemas.py, tools/analyze_duplicates.py, tools/propose_repairs.py, tools/prompt_to_ir.py, tools/build_wiki_seed.py.
  3. ci — .github/workflows/audit.yml + docs/ci-and-quality-gates.md.
  4. catalog — catalog/unified-workflows.index.json + refreshed catalog/workflows.index.json.
  5. reports — generated reports + reports/intelligence-core-v1.*.
  6. wiki-seed — wiki/generated/ new folder.
  7. readme — README.md additive section last.

Run python tools/audit_workflows.py && python tools/validate_schemas.py once more right before tagging — both should report all gates passed and 0 possible secret findings.

Final judgment

READY FOR HUMAN DIFF REVIEW.

Not committed. Not pushed. No branch created.