Skip to content

[docs] Add OB1 catalog generator + committed ob1-catalog.json#187

Open
justfinethanku wants to merge 1 commit into
contrib/jonathanedwards/ob1-compatibility-catalogfrom
contrib/jonathanedwards/ob1-catalog-generator
Open

[docs] Add OB1 catalog generator + committed ob1-catalog.json#187
justfinethanku wants to merge 1 commit into
contrib/jonathanedwards/ob1-compatibility-catalogfrom
contrib/jonathanedwards/ob1-catalog-generator

Conversation

@justfinethanku
Copy link
Copy Markdown
Collaborator

Summary

  • Ships scripts/build_catalog.py — the generator that scans every non-template contribution folder and emits a site-ready JSON artifact with rewritten READMEs, resolved dependency edges, and per-entry compatibility metadata.
  • Commits the first resources/ob1-catalog.json artifact (59 entries, ~570 KB).
  • Wires a drift check into .github/workflows/ob1-gate.yml — any PR that touches contribution content, the generator, or the artifact must also update the artifact or the gate fails.

Stacked on PR #186

This PR's base is contrib/jonathanedwards/ob1-compatibility-catalog (PR #186) because the generator relies on the new requires.open_brain string enum. Merge #186 first, then retarget this to main and merge.

Why the generator runs at commit time (not site build time)

  • Deterministic site builds (natejones.com just reads one JSON file).
  • Generator-time validation catches repo semantics (missing deps, broken relative links, duplicate site paths) — the site loader only validates artifact shape.
  • The gate's drift check means a contributor cannot land a contribution without also regenerating and committing the artifact.

Generator-time validation (the contract)

  1. category in metadata.json matches the folder category.
  2. requires.open_brain is one of required | optional.
  3. learning_order is only present on extensions.
  4. Every requires_skills / requires_primitives slug resolves to a real contribution.
  5. No two entries share the same site_path.
  6. Every relative intra-repo link in a contribution README resolves to an existing file.

The loader on the site side only has to validate artifact shape, not repo semantics.

Link rewriting rules

  • Relative links to contribution folders / READMEs → /ob1/<category>/<slug>
  • Links to category index folders / READMEs → /ob1/<category>
  • Deeper relative files inside a contribution folder → raw GitHub blob URL
  • Image / asset refs → raw.githubusercontent.com (so they render on the site)
  • Other intra-repo paths (e.g. docs/, root markdown) → GitHub blob/tree URL
  • Absolute URLs, anchors, and mailto: / tel: — pass through untouched

Site loader precedence (documented here; implemented in the site PR)

  • OB1_CATALOG_LOCAL_PATH env var takes precedence when set (local dev override)
  • Otherwise fetch from raw.githubusercontent.com/.../main/resources/ob1-catalog.json
  • Missing or invalid artifact must fail the site build

Pre-existing fix included

recipes/email-history-import/README.md linked to primitives/content-fingerprint-dedup/ — that directory does not exist; the fingerprint dedup logic lives in recipes/content-fingerprint-dedup/. The generator's strict relative-link check flagged it. Fixed the link rather than loosening the rule, per the plan.

Catalog breakdown (first artifact)

Category Count Required Optional
recipes 29 29 0
schemas 1 1 0
dashboards 2 2 0
integrations 3 3 0
skills 13 4 9
primitives 5 5 0
extensions 6 6 0

Test plan

  • Generator emits a clean ob1-catalog.json on the current tree (59 entries, no validation errors).
  • --check mode returns 0 on the committed artifact.
  • Reverse dependency graph is populated (verified: skills/auto-capture shows recipes/auto-capture as a reverse dep).
  • Sample optional skill (skills/panning-for-gold) serializes with compatibility.open_brain = "optional".
  • Sample required skill (skills/auto-capture) serializes with compatibility.open_brain = "required".
  • Link rewriting verified against recipes/auto-capture README: skills/auto-capture//ob1/skills/auto-capture, docs/01-getting-started.md → GitHub blob URL.
  • Post-merge: gate drift check triggers correctly on the next contribution PR.

In-flight contribution impact

Any open contribution PR that adds a new folder under recipes/, skills/, etc. will now need resources/ob1-catalog.json regenerated in the same PR. The error message includes the exact command to run. Maintainer-side fix on merge is fine for current volume.

🤖 Generated with Claude Code

Ships the site-ready catalog artifact that the natejones.com /ob1
directory will load, plus the generator script and a PR gate drift
check that keeps the artifact in sync with repo content.

Pipeline:
- scripts/build_catalog.py discovers every non-template contribution
  under the seven canonical category folders, loads metadata.json,
  rewrites intra-repo links in README.md, resolves requires_skills /
  requires_primitives into forward and reverse dependency edges, and
  emits resources/ob1-catalog.json.
- .github/workflows/ob1-gate.yml now runs the generator in --check
  mode whenever a PR touches contribution content, the generator, or
  the artifact. A drift failure tells the contributor exactly which
  command to run.

Generator-time validation (runs before artifact is emitted):
- category in metadata.json matches the folder category
- requires.open_brain is one of required | optional
- learning_order is only set on extensions
- every requires_skills / requires_primitives slug resolves to a
  real contribution
- no two entries share the same site_path
- every relative intra-repo link in a contribution README resolves
  to an existing file in the repo

Link rewriting:
- relative links to contribution folders or their READMEs become
  /ob1/<category>/<slug>
- links to category index folders or READMEs become /ob1/<category>
- deeper relative files inside contribution folders go to the raw
  GitHub blob URL
- image / asset references go to raw.githubusercontent.com
- other intra-repo links (docs/, root markdown) go to blob/tree URLs
- absolute URLs, anchors, and mailto: / tel: schemes pass through

Site loader precedence (documented here for PR3 in the site repo):
- OB1_CATALOG_LOCAL_PATH env var takes precedence when set
- otherwise fetch from raw.githubusercontent.com on main
- missing or invalid artifact must fail the site build

Pre-existing fix included:
- recipes/email-history-import/README.md linked to a non-existent
  primitives/content-fingerprint-dedup path. The dedup logic lives in
  recipes/content-fingerprint-dedup. The generator's strict link
  check flagged it, so fixed the link instead of loosening the rule.

Output: 59 catalog entries across all seven categories
(recipes 29, skills 13, extensions 6, primitives 5, integrations 3,
dashboards 2, schemas 1) — 4 required + 9 optional in skills, all
other categories required for v1.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@alanshurafa alanshurafa added area: docs Review area: documentation review: needs-author-response Contributor response or clarification needed alan-reviewed Reviewed by Alan Shurafa in Community Reviewer role labels May 20, 2026
@alanshurafa
Copy link
Copy Markdown
Collaborator

Thanks for the contribution. The catalog generator (scripts/build_catalog.py) producing a site-ready ob1-catalog.json is a reasonable idea, and the drift-check approach (the gate fails if the artifact goes stale) is a sound way to keep it in sync.

Two things, though. This PR is stacked on #186 (contrib/jonathanedwards/ob1-compatibility-catalog), so its mergeability follows #186 — and #186 has an open scope question (the breaking metadata.schema.json change across 71 files; I recommended splitting it). The catalog generator and the compatibility contract are really one feature in two stacked PRs, so they are best handled together, with #186's scope settled first. Separately, #187 modifies .github/workflows/ob1-gate.yml to add the drift check — a workflow change, outside what I can sign off on, and the gate file is already in flux across #308 and #289.

Recommend handling #186 and #187 together; this one effectively waits on #186.

— Alan (community reviewer; non-binding)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

alan-reviewed Reviewed by Alan Shurafa in Community Reviewer role area: docs Review area: documentation review: needs-author-response Contributor response or clarification needed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants