Skip to content

demo.md

Latest commit

 

History

History
169 lines (119 loc) · 5.28 KB

demo.md

File metadata and controls

169 lines (119 loc) · 5.28 KB
Raw

Local Demo Walkthrough

This page walks through what you can do with this repository on a developer laptop. Every step is local, standard-library Python, and does not require n8n to be installed.

If any step requires more than what is listed here, that is a bug — please open an issue with the exact command and the output.

Prerequisites

  • Python 3.12 or newer (the tools target the standard library only).
  • A shell that can run python tools/<name>.py.
  • About 250 MB of free disk for generated reports if you run every demo.

1. Audit demo

Read-only catalog + secret scan over the entire repository.

python tools/audit_workflows.py

Expected output:

  • catalog/workflows.index.json is regenerated.
  • reports/workflow-audit.json and reports/workflow-audit.md are written.
  • The console line ends with possible secret findings followed by a count. Current expected count: 0.

What the audit does:

  • Parses every *.json under the repository.
  • Catalogues n8n-shaped workflows.
  • Runs a conservative secret-pattern scan and redacts every match before anything is written to disk.

What it does not do:

  • It does not execute any workflow.
  • It does not call any external service.
  • It does not modify any source workflow file.

2. Catalog search demo

The unified catalog merges the existing library and the generated pack with deterministic quality scores.

python tools/build_unified_catalog.py
python tools/search_workflows.py --query "lead qualification" --top 5

Expected output (search):

  • A ranked list of up to 5 matching catalog entries, each showing title, category, integrations, trigger, source path, risk level, and the quality flags that contributed to its review status.

What it proves:

  • Catalog entries are addressable and searchable.
  • The unified catalog can answer prompts like "find me lead-qualification workflows" without an LLM.

What it does not prove:

  • That any returned workflow runs. That is what the runtime proof step is for — and even then, only with placeholders.

3. Prompt-to-n8n proof demo

The end-to-end deterministic loop: prompt → Universal Workflow IR → n8n workflow JSON → static n8n compatibility validation → repair proposal (if needed) → learning event (if useful) → human review queue.

python tools/prompt_to_n8n.py "Create a workflow that receives website leads, scores them, saves them to CRM, and alerts Slack."

Expected output:

  • A new directory under reports/runtime-proof/<slug>/ containing:
    • prompt.txt
    • workflow.ir.json (draft Universal Workflow IR)
    • workflow.n8n.json (n8n-shaped workflow with safe placeholders only)
    • validation.json and validation.md (static validator output)
    • README.md (human-readable proof summary)
  • Console line ending with validation=PASS errors=0 warnings=N nodes=N.

Run the validator directly on the exported workflow if you want to see the report on its own:

python tools/validate_n8n_workflow.py reports/runtime-proof/<slug>/workflow.n8n.json

Then generate downstream artefacts:

python tools/propose_runtime_repair.py reports/runtime-proof/<slug>/
python tools/create_learning_event.py reports/runtime-proof/<slug>/

This adds repair-proposals.{json,md} and learning-events.{json,md} to the slug directory. Both files conform to the project schemas and both carry the "requires human review / not applied" markers.

What this proves:

  • The static loop is deterministic and produces structurally valid n8n workflow JSON with no real credentials and no real URLs.

What this does not prove:

  • That the workflow imports into n8n.
  • That any node executes correctly.
  • That the prompt was understood with anything more than keyword rules.

4. Review queue demo

python tools/build_review_queue.py

Expected output:

  • reports/review-queue.json and reports/review-queue.md are (re)generated.
  • The console line ends with total=N types={...} sev={...}.

What it does:

  • Aggregates pre-existing catalog repair proposals, runtime-proof repair proposals, learning events, high-risk workflows, duplicate candidates, generated-pack-needs-behavioural-test items, and per-runtime-proof review items into a single human review queue.
  • Every item is status: pending, humanReviewRequired: true.

What it does not do:

  • It does not approve or apply anything.
  • It does not modify any workflow.
  • It does not change the curated wiki.

See docs/review-queue.md for what each item type means and how to triage it.

5. Limitations of this demo

  • Static validation only. No node is executed. No n8n instance is contacted.
  • All URLs and credentials in generated output are placeholders.
  • Prompt-to-IR is keyword-rule. Synonyms and plurals are likely to be missed by design.
  • Only n8n export is implemented. Dify / LangGraph / Make / Zapier are documented but not implemented.
  • The reports written by these tools are reviewable starting points, not certified outputs.

6. One-shot end-to-end check

python tools/final_readiness_check.py

This orchestrates the audit, validators, catalog build, runtime proof smoke, public-claims audit, and review queue build. It exits 0 only if the hard gates pass. See docs/maintainer-release-checklist.md.