Skip to content

Commit 53e6611

Browse files
feat(ecosystem-sync): report-only drift orchestrator + ecosystem.json (safe slice) (#1)
* feat(ecosystem-sync): report-only drift orchestrator + ecosystem.json manifest (safe slice) Stands up the ecosystem-sync home as a REPORT-ONLY safe slice: - ecosystem.json — manifest of the 6 IEP repos (core, lab, audit-harness, j-rig, rollout-gate, dashboard) with kind + version_source + pinned_version. Lists claude-code-plugins under "excluded" (bespoke in-flight CI/CD — coordinate first, sequence last; never bolt generic automation on). - scripts/ecosystem-drift.py — reads the manifest, resolves live upstream (npm / git tag), reports drift. Opens NO PRs, mutates nothing, always exits 0 (advisory). - .github/workflows/ecosystem-drift.yml — weekly + on-manifest-change; contents:read only, so it structurally cannot open PRs. Writes report to the run summary. - README — documents the manifest, the report-only guarantee, and the exclusion. Deferred (separate, explicitly-reviewed change): live auto-PR authority and folding in claude-code-plugins. * harden(ecosystem-drift): UA header + defensive guards per review Address 6 medium-priority review findings on the report-only drift checker: - centralize a descriptive User-Agent in _get_json (avoids GitHub/npm rejecting/throttling the default urllib UA) - guard empty package/gh args in npm_latest + newest_tag - resolve() reads package/gh via .get() (no KeyError on malformed manifest) - main() / excluded loop read name via .get(..., "unknown") Safe-slice guarantees unchanged: read-only, opens no PRs, always exits 0. --------- Co-authored-by: jeremylongshore <jeremylongshore@users.noreply.github.com>
1 parent f3227e6 commit 53e6611

4 files changed

Lines changed: 273 additions & 0 deletions

File tree

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
name: Ecosystem drift (report-only)
2+
3+
# SAFE SLICE: this workflow only REPORTS drift. It has read-only permissions and
4+
# opens no PRs. Granting write/PR authority is a separate, explicitly-reviewed change.
5+
6+
on:
7+
schedule:
8+
- cron: "0 6 * * 1" # Mondays 06:00 UTC — weekly drift report
9+
workflow_dispatch: {}
10+
push:
11+
branches: [main]
12+
paths:
13+
- "ecosystem.json"
14+
- "scripts/ecosystem-drift.py"
15+
- ".github/workflows/ecosystem-drift.yml"
16+
17+
permissions:
18+
contents: read # read-only by design — no PR/commit authority
19+
20+
concurrency:
21+
group: ecosystem-drift-${{ github.ref }}
22+
cancel-in-progress: true
23+
24+
jobs:
25+
drift:
26+
name: Report ecosystem version drift
27+
runs-on: ubuntu-latest
28+
steps:
29+
- uses: actions/checkout@v4
30+
- uses: actions/setup-python@v5
31+
with:
32+
python-version: "3.12"
33+
- name: Run report-only drift checker
34+
env:
35+
GITHUB_TOKEN: ${{ github.token }} # for GitHub API rate limits on tag lookups (read-only)
36+
run: python3 scripts/ecosystem-drift.py

README.md

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -107,4 +107,28 @@ You don't have to trust any of it — that's the point. Verify it yourself.
107107

108108
---
109109

110+
## Ecosystem manifest + drift report (report-only)
111+
112+
[`ecosystem.json`](./ecosystem.json) is the machine-readable map of the six platform
113+
repos — each repo's kind, its release channel (npm or git tag), and the **pinned
114+
version** the platform considers current. [`scripts/ecosystem-drift.py`](./scripts/ecosystem-drift.py)
115+
reads the manifest and reports which repos have drifted behind their live upstream. The
116+
[`ecosystem-drift`](./.github/workflows/ecosystem-drift.yml) workflow runs it weekly and
117+
on manifest changes, writing the report to the run summary.
118+
119+
**This is a report-only safe slice.** The checker has read-only permissions, opens no
120+
PRs, and mutates nothing — it always exits 0 (drift is advisory). Acting on drift means
121+
re-verifying upstream and bumping `pinned_version` in `ecosystem.json` via a reviewed PR.
122+
123+
`claude-code-plugins` is deliberately listed under `excluded`, not automated: it has
124+
bespoke in-flight CI/CD, so any future sync automation must be coordinated with that
125+
pipeline first and sequenced last. Turning on auto-PR (and folding in the excluded repo)
126+
is a separate, explicitly-reviewed change — not a config flip.
127+
128+
```bash
129+
python3 scripts/ecosystem-drift.py # prints the drift table; opens nothing
130+
```
131+
132+
---
133+
110134
<sub>Intent Solutions · [intentsolutions.io](https://intentsolutions.io) · Apache-2.0</sub>

ecosystem.json

Lines changed: 72 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,72 @@
1+
{
2+
"manifest_version": "0.1.0",
3+
"description": "Intent Eval Platform ecosystem manifest. Single source of truth for the report-only drift orchestrator (scripts/ecosystem-drift.py): which repos compose the platform, each repo's kind, and the pinned version the orchestrator compares against live upstream. SAFE SLICE — report-only: the orchestrator NEVER opens PRs in this version. Live auto-PR and inclusion of claude-code-plugins are deferred pending coordination (see the 'excluded' block and README).",
4+
"generated": "2026-06-06",
5+
"automation": {
6+
"mode": "report-only",
7+
"auto_pr": false,
8+
"note": "Auto-PR is intentionally disabled. Turning it on is a separate, explicitly-reviewed change — not a config flip in this PR."
9+
},
10+
"repos": [
11+
{
12+
"name": "intent-eval-core",
13+
"gh": "jeremylongshore/intent-eval-core",
14+
"kind": "node-package",
15+
"version_source": "npm",
16+
"package": "@intentsolutions/core",
17+
"pinned_version": "0.2.0",
18+
"automation": "report-only"
19+
},
20+
{
21+
"name": "intent-eval-lab",
22+
"gh": "jeremylongshore/intent-eval-lab",
23+
"kind": "docs-methodology",
24+
"version_source": "git-tag",
25+
"pinned_version": "v0.2.0",
26+
"automation": "report-only"
27+
},
28+
{
29+
"name": "intent-audit-harness",
30+
"gh": "jeremylongshore/intent-audit-harness",
31+
"kind": "polyglot-cli",
32+
"version_source": "npm",
33+
"package": "@intentsolutions/audit-harness",
34+
"pinned_version": "1.1.5",
35+
"automation": "report-only"
36+
},
37+
{
38+
"name": "j-rig-skill-binary-eval",
39+
"gh": "jeremylongshore/j-rig-skill-binary-eval",
40+
"kind": "node-monorepo",
41+
"version_source": "git-tag",
42+
"pinned_version": "v1.1.0",
43+
"automation": "report-only"
44+
},
45+
{
46+
"name": "intent-rollout-gate",
47+
"gh": "jeremylongshore/intent-rollout-gate",
48+
"kind": "github-action",
49+
"version_source": "git-tag",
50+
"pinned_version": "v0.0.1",
51+
"automation": "report-only"
52+
},
53+
{
54+
"name": "intent-eval-dashboard",
55+
"gh": "jeremylongshore/intent-eval-dashboard",
56+
"kind": "web-dashboard",
57+
"version_source": "git-tag",
58+
"pinned_version": null,
59+
"automation": "report-only",
60+
"note": "No release tag yet; orchestrator reports 'untagged' rather than drift."
61+
}
62+
],
63+
"excluded": [
64+
{
65+
"name": "claude-code-plugins",
66+
"gh": "jeremylongshore/claude-code-plugins",
67+
"kind": "marketplace",
68+
"automation": "EXCLUDED",
69+
"reason": "Has bespoke in-flight CI/CD that tests plugins. Any sync automation must be studied and coordinated with that pipeline FIRST — never bolt generic automation on top of it. When ecosystem-sync gains write authority, claude-code-plugins is sequenced LAST and only after explicit coordination. The orchestrator must skip this entry entirely."
70+
}
71+
]
72+
}

scripts/ecosystem-drift.py

Lines changed: 141 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,141 @@
1+
#!/usr/bin/env python3
2+
"""
3+
ecosystem-drift — REPORT-ONLY drift checker for the Intent Eval Platform.
4+
5+
Reads ecosystem.json and, for each repo whose automation is "report-only",
6+
resolves the live upstream version (npm latest, or newest git tag) and compares
7+
it to the manifest's pinned_version. Prints a markdown drift report.
8+
9+
SAFE-SLICE GUARANTEES (do not weaken without an explicit, reviewed change):
10+
* NEVER opens a PR, pushes a commit, or mutates any repo. Read-only.
11+
* NEVER touches repos in the manifest's "excluded" block (claude-code-plugins) —
12+
they are listed for visibility and skipped by design.
13+
* ALWAYS exits 0. Drift is advisory; this script has no exit authority.
14+
15+
Stdlib only. Network: registry.npmjs.org (no auth) + api.github.com (optional
16+
GITHUB_TOKEN from env for higher rate limits). Soft-fails per repo.
17+
"""
18+
import json
19+
import os
20+
import sys
21+
import urllib.request
22+
import urllib.error
23+
24+
HERE = os.path.dirname(os.path.abspath(__file__))
25+
MANIFEST = os.path.join(HERE, "..", "ecosystem.json")
26+
27+
28+
# Some registries/APIs (notably api.github.com) reject the default Python
29+
# urllib User-Agent. Send a descriptive UA on every request, centrally.
30+
_UA = "ecosystem-drift (+https://github.com/intent-solutions-io/intent-eval-platform)"
31+
32+
33+
def _get_json(url, headers=None, timeout=15):
34+
hdrs = {"User-Agent": _UA}
35+
if headers:
36+
hdrs.update(headers)
37+
req = urllib.request.Request(url, headers=hdrs)
38+
with urllib.request.urlopen(req, timeout=timeout) as r:
39+
return json.load(r)
40+
41+
42+
def npm_latest(pkg):
43+
if not pkg:
44+
return None
45+
try:
46+
return _get_json(f"https://registry.npmjs.org/{pkg}/latest").get("version")
47+
except (urllib.error.URLError, ValueError, TimeoutError):
48+
return None
49+
50+
51+
def newest_tag(gh):
52+
if not gh:
53+
return None
54+
headers = {"Accept": "application/vnd.github+json"} # UA centralized in _get_json
55+
token = os.environ.get("GITHUB_TOKEN") or os.environ.get("GH_TOKEN")
56+
if token:
57+
headers["Authorization"] = f"Bearer {token}"
58+
try:
59+
tags = _get_json(f"https://api.github.com/repos/{gh}/tags", headers=headers)
60+
return tags[0]["name"] if tags else None
61+
except (urllib.error.URLError, ValueError, TimeoutError, KeyError, IndexError):
62+
return None
63+
64+
65+
def _norm(v):
66+
return v.lstrip("v") if isinstance(v, str) else v
67+
68+
69+
def resolve(repo):
70+
src = repo.get("version_source")
71+
if src == "npm":
72+
return npm_latest(repo.get("package"))
73+
if src == "git-tag":
74+
return newest_tag(repo.get("gh"))
75+
return None
76+
77+
78+
def status_for(pinned, latest):
79+
if pinned is None and latest is None:
80+
return "untagged (no pin / no release)"
81+
if latest is None:
82+
return "unknown (lookup failed)"
83+
if pinned is None:
84+
return "untagged (no pin)"
85+
if _norm(pinned) == _norm(latest):
86+
return "current"
87+
return "BEHIND"
88+
89+
90+
def main():
91+
with open(MANIFEST, encoding="utf-8") as f:
92+
doc = json.load(f)
93+
94+
rows, behind = [], 0
95+
for repo in doc.get("repos", []):
96+
if repo.get("automation") != "report-only":
97+
continue
98+
latest = resolve(repo)
99+
st = status_for(repo.get("pinned_version"), latest)
100+
if st == "BEHIND":
101+
behind += 1
102+
rows.append((repo.get("name", "unknown"), repo.get("version_source", "—"),
103+
str(repo.get("pinned_version")), str(latest), st))
104+
105+
lines = []
106+
lines.append(f"# Ecosystem drift report (report-only) — manifest v{doc.get('manifest_version')}")
107+
lines.append("")
108+
lines.append(f"Mode: **{doc.get('automation', {}).get('mode')}** · auto-PR: "
109+
f"**{doc.get('automation', {}).get('auto_pr')}** · repos behind: **{behind}**")
110+
lines.append("")
111+
lines.append("| Repo | Source | Pinned | Latest upstream | Status |")
112+
lines.append("|------|--------|--------|-----------------|--------|")
113+
for name, src, pinned, latest, st in rows:
114+
lines.append(f"| {name} | {src} | {pinned} | {latest} | {st} |")
115+
excluded = doc.get("excluded", [])
116+
if excluded:
117+
lines.append("")
118+
lines.append("## Excluded from automation (skipped by design)")
119+
for ex in excluded:
120+
lines.append(f"- **{ex.get('name', 'unknown')}** — {ex.get('reason', 'excluded')}")
121+
lines.append("")
122+
lines.append("_Advisory only. This report opens no PRs and mutates nothing. "
123+
"To act on drift, re-verify upstream and bump `pinned_version` in "
124+
"`ecosystem.json` via a reviewed PR._")
125+
report = "\n".join(lines)
126+
127+
print(report)
128+
summary = os.environ.get("GITHUB_STEP_SUMMARY")
129+
if summary:
130+
try:
131+
with open(summary, "a", encoding="utf-8") as f:
132+
f.write(report + "\n")
133+
except OSError:
134+
pass
135+
136+
# Advisory: never any exit-code authority.
137+
sys.exit(0)
138+
139+
140+
if __name__ == "__main__":
141+
main()

0 commit comments

Comments
 (0)