diff --git a/README.md b/README.md index 68cda9f..29098d5 100644 --- a/README.md +++ b/README.md @@ -36,6 +36,12 @@ flowchart LR See [Architecture Diagrams](docs/architecture.md), [Design](DESIGN.md), [Hosted OpenCoven](HOSTED.md), [Familiar Contract](FAMILIAR-CONTRACT.md), [Roadmap](ROADMAP.md), and [Hosted vs self-hosted](docs/hosted-vs-self-hosted.md) for the operational plan. +**Want to see the whole loop run?** `examples/demo/run-demo.sh` drives the +real adapter through the full operating loop — issue assignment, Check Run, +edited-in-place status comment, maintainer steering commands, permission +gate, familiar-voice draft PR — offline and self-verifying in about a minute. +[docs/demo.md](docs/demo.md) narrates it. + --- ## Architecture @@ -146,6 +152,7 @@ duplicate comments. | Label trigger | Implemented | Routes configured `trigger_labels` such as `coven:fix`. | | Maintainer command protocol | Implemented | Typed `@familiar ` grammar; casual mentions ignored; write-access gate; self-comments never re-trigger. | | Marker-backed status comments | Implemented | One edited-in-place status surface per issue/PR; no duplicate bot comments. | +| Reference demo of the operating loop | Implemented | Offline, self-verifying replay of the full loop with a real adapter binary — see [docs/demo.md](docs/demo.md). | | PR lifecycle review trigger | Implemented | Policy-gated auto-review on opened/synchronize/reopened/ready_for_review plus label opt-in; familiar-authored PRs are never auto-reviewed. | | Push / commit review trigger | Partial | Events parsed and typed with fixtures; execution lane needs headless contract v3. | | GitHub App installation tokens | Implemented | Mints installation access tokens from the App private key. | diff --git a/docs/demo.md b/docs/demo.md new file mode 100644 index 0000000..cceb434 --- /dev/null +++ b/docs/demo.md @@ -0,0 +1,116 @@ +# Reference demo: the operating loop, end to end + +This is the ClawSweeper-style reference demo of the coven-github operating +loop (issue #19): the **real adapter binary**, driven by signed webhook +deliveries, publishing to an in-memory GitHub API stub through a +contract-conformant fake `coven-code` runtime. No network, no GitHub App +registration, no credentials — one command, about a minute, self-verifying. + +```bash +examples/demo/run-demo.sh +``` + +A green exit code *is* the demo's claim: every property below is asserted +programmatically against the stub's recorded state, not eyeballed. Set +`KEEP=1` to keep the scratch directory (config, server log, stub log) for +inspection. + +## Why "ClawSweeper-style" + +ClawSweeper is the reference pattern for conservative GitHub automation in the +OpenClaw ecosystem (see [`DESIGN.md`](../DESIGN.md)): narrow promises, durable +state, marker-backed comments edited in place, explicit maintainer commands, +and deterministic gates before mutation. This demo shows coven-github honoring +each of those disciplines with its own machinery. + +## What runs + +| Piece | Real or stand-in | Role | +|---|---|---| +| `coven-github serve` | **Real** — the same binary you self-host | Webhook receiver, command router, worker pool, publication | +| Webhook deliveries | Real HMAC-SHA256 signatures, GitHub payload shapes | Drive every act | +| GitHub API | [`github-stub.py`](../examples/demo/github-stub.py) — in-memory stand-in | Records every mutation in an audit trail | +| `coven-code --headless` | [`fake-coven-code`](../examples/demo/fake-coven-code) — headless contract v2 conformant | Fabricates a familiar-voice fix instead of running a model | +| App credentials | Throwaway RSA key + random webhook secret, generated per run | Prove the real JWT → installation-token path | + +The stub answers exactly the API surface the adapter uses — scoped token +minting, repo metadata, branch resolution, collaborator permission, Check +Runs, issue comments, pull requests — and exposes its world state at +`/_demo/state` and the mutation log at `/_demo/audit`. + +## The acts + +**Act 1 — issue assigned: the full loop.** `octocat` assigns issue #42 to +`@coven-cody`. The adapter mints an **orchestration** token, resolves the +default branch and head SHA from live state, creates a Check Run against that +immutable SHA, posts the one marker-backed status comment, flips the check to +`in_progress`, mints a separate **agent-git** token injected only via +`COVEN_GIT_TOKEN` (the fake runtime verifies the brief itself is tokenless), +runs the session, then mints a **publication** token and opens a draft PR back +to the issue — in Cody's voice. Asserted: one comment, edited in place to +`Status: done`; Check Run concluded `success`; PR is a draft; three distinct +token scopes minted. + +**Act 2 — casual mention.** "thanks @coven-cody, great work on this!" triggers +*nothing*. Asserted: the audit trail did not grow by a single API call. + +**Act 3 — self-trigger loop guard.** The familiar's own comment containing +`@coven-cody status` is ignored. Asserted: audit unchanged. + +**Act 4 — unknown verb.** `@coven-cody explain` is a command-position mention +with an unknown verb, so the familiar replies with the real command list +instead of guessing. Asserted: the clarification *edited the same status +comment* — still exactly one surface. + +**Act 5 — steering: `status`.** Answered from the durable task store (the same +state Cave polls), listing the issue's tasks and their lifecycle states. + +**Act 6 — permission gate.** `mallory` (read-only) comments +`@coven-cody retry`. The worker checks collaborator permission *pre-flight* +and declines on the status surface. Asserted: `Status: declined` and **no new +Check Run** — no session was spent below the write-access bar. + +**Act 7 — steering: `retry`.** `octocat` (admin) retries. A second full run +executes: second Check Run to `success`, second draft PR. Asserted: **still +exactly one status comment** — repeated runs never stack duplicates. + +## The closing surfaces + +After the acts, the demo prints two oversight views: + +- **The audit trail** — every GitHub mutation in order, attributed to the + token role that performed it. You can watch the authority split from + issue #4 in action: `orchestration` drives checks and comments, + `publication` opens the PR, `agent-git` never touches the API at all. +- **The Cave view** — `GET /api/github/tasks`, the adapter's task API that + the CovenCave dashboard (issue #18) polls: task ids, branches, PR links, + Check Run links, session ids, lifecycle status. + +Cave *intervention* — pausing or steering a live session from the dashboard — +lands with #18; today the oversight loop closes through the maintainer +commands demonstrated above plus the session deep links on every Check Run +and status comment. + +## Sample audit trail + +```text + 1 app-jwt mint orchestration token (repo-scoped: demo-service) + 2 orchestration read repo metadata OpenCoven/demo-service + 3 orchestration resolve branch 'main' head SHA + 4 orchestration create check run 1001 'Cody — Fix issue #42: …' (queued) + 5 orchestration list #42 comments (0 found) + 6 orchestration post comment 5001 on #42 + 7 orchestration check run 1001 -> in_progress ('Running') + 8 app-jwt mint agent-git token (repo-scoped: demo-service) + 9 app-jwt mint publication token (repo-scoped: demo-service) + 10 publication open draft PR #101 (cody/fix-issue-42 -> main) + 11 orchestration list #42 comments (1 found) + 12 orchestration edit comment 5001 in place (edit #1: 'Status: done') + 13 orchestration check run 1001 -> success ('Done') +``` + +## Requirements + +`cargo`, `python3` (stdlib only), `openssl`, `curl`. The script builds the +adapter with `cargo build -p coven-github`, picks free ports, and cleans up +after itself (scratch dir is kept on failure, or with `KEEP=1`). diff --git a/examples/demo/__pycache__/fake-coven-codecpython-314.pyc b/examples/demo/__pycache__/fake-coven-codecpython-314.pyc new file mode 100644 index 0000000..99b3af0 Binary files /dev/null and b/examples/demo/__pycache__/fake-coven-codecpython-314.pyc differ diff --git a/examples/demo/__pycache__/github-stub.cpython-314.pyc b/examples/demo/__pycache__/github-stub.cpython-314.pyc new file mode 100644 index 0000000..f0c505f Binary files /dev/null and b/examples/demo/__pycache__/github-stub.cpython-314.pyc differ diff --git a/examples/demo/fake-coven-code b/examples/demo/fake-coven-code new file mode 100755 index 0000000..22f807b --- /dev/null +++ b/examples/demo/fake-coven-code @@ -0,0 +1,137 @@ +#!/usr/bin/env python3 +"""Contract-conformant stand-in for `coven-code --headless` (demo, issue #19). + +Speaks headless contract v2 (`docs/headless-contract.md`): reads the tokenless +session brief from `--context`, writes the structured result envelope to +`--output`, and exits 0. Instead of running a real agent session, it fabricates +a small, plausible fix in the familiar's voice so the adapter's publication +path (Check Run, status comment, draft PR) can be demonstrated offline. + +It also proves two security invariants of the real integration on every run: + + * the brief is tokenless — no credential material appears in the JSON; + * git authority arrives only via the COVEN_GIT_TOKEN environment variable. +""" + +import json +import os +import sys + + +def parse_args(argv): + args = {"headless": False, "context": None, "output": None} + it = iter(argv) + for arg in it: + if arg == "--headless": + args["headless"] = True + elif arg == "--context": + args["context"] = next(it, None) + elif arg == "--output": + args["output"] = next(it, None) + if not (args["headless"] and args["context"] and args["output"]): + sys.exit("usage: fake-coven-code --headless --context BRIEF --output RESULT") + return args + + +def no_review(): + return { + "mode": "none", + "evidence_status": "not_applicable", + "reviewed_files": [], + "supporting_files": [], + "findings": [], + "tests_run": [ + { + "command": "cargo test -p auth refresh", + "status": "passed", + "output_summary": "9/9 passing", + } + ], + "no_findings_reason": None, + "limitations": [], + } + + +def main(): + args = parse_args(sys.argv[1:]) + with open(args["context"], encoding="utf-8") as f: + raw_brief = f.read() + brief = json.loads(raw_brief) + + if brief.get("contract_version") != "2": + sys.exit(f"unsupported brief contract_version: {brief.get('contract_version')}") + + # Invariant checks (issue #4): tokenless brief, env-injected git authority. + git_token = os.environ.get("COVEN_GIT_TOKEN", "") + if not git_token: + sys.exit("COVEN_GIT_TOKEN missing — the worker must inject git authority") + if git_token in raw_brief: + sys.exit("SECURITY: git token leaked into the session brief") + print( + "fake-coven-code: brief is tokenless; git authority arrived via env only", + file=sys.stderr, + ) + + familiar = brief["familiar"]["display_name"] + task = brief["task"] + kind = task["kind"] + + if kind == "fix_issue": + number = task["issue_number"] + branch = f"cody/fix-issue-{number}" + summary = ( + f"Fixed the OAuth token refresh by adding a 60-second " + f"clock-skew buffer (issue #{number})." + ) + pr_body = ( + f"## Hey, I'm {familiar} \U0001f43e\n\n" + f"Issue #{number} traced to the OAuth refresh path not accounting " + f"for clock skew between the client and the auth server. I added a " + f"60-second buffer to the expiry check and regression tests for " + f"both skew directions.\n\n" + f"**Changed:** `src/auth/refresh.rs` (+14 / -3)\n" + f"**Tests:** 9/9 passing (2 new regression cases)\n\n" + f"I kept the diff scoped to the refresh path; the token store " + f"itself is untouched. Shout if you want the buffer configurable." + ) + commits = [ + { + "sha": "f1e2d3c4b5a6", + "message": f"fix: add clock-skew buffer to OAuth refresh (#{number})", + } + ] + files = ["src/auth/refresh.rs", "src/auth/refresh_test.rs"] + elif kind == "address_review_comment": + number = task["pr_number"] + branch = f"cody/pr-{number}-feedback" + summary = f"Addressed review feedback on PR #{number}." + pr_body = ( + f"## {familiar} here \U0001f43e\n\n" + f"Follow-up for the review feedback on #{number} — renamed the " + f"helper and added the missing error-path test." + ) + commits = [ + {"sha": "0a1b2c3d4e5f", "message": "refactor: address review feedback"} + ] + files = ["src/auth/refresh.rs"] + else: + sys.exit(f"demo runtime does not handle task kind: {kind}") + + result = { + "contract_version": "2", + "status": "success", + "branch": branch, + "commits": commits, + "files_changed": files, + "summary": summary, + "pr_body": pr_body, + "review": no_review(), + "exit_reason": None, + } + with open(args["output"], "w", encoding="utf-8") as f: + json.dump(result, f, indent=2) + print(f"fake-coven-code: wrote result for {kind} -> {branch}", file=sys.stderr) + + +if __name__ == "__main__": + main() diff --git a/examples/demo/github-stub.py b/examples/demo/github-stub.py new file mode 100755 index 0000000..8e1a40f --- /dev/null +++ b/examples/demo/github-stub.py @@ -0,0 +1,296 @@ +#!/usr/bin/env python3 +"""In-memory GitHub API stub for the reference demo (issue #19). + +Stands in for api.github.com so the full operating loop — scoped token +minting, Check Runs, marker-backed status comments, draft PRs, and the +maintainer permission gate — runs end to end with zero network access and +zero real credentials. + +The stub keeps one world of state and records every API call in an audit +trail. Two demo-only endpoints expose it: + + GET /_demo/state full world state as JSON (assertions read this) + GET /_demo/audit the audit trail as JSON + +Run: python3 github-stub.py --port 8091 +""" + +import argparse +import json +import re +import threading +import time +from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer + +# Who may command the familiar. Everyone else defaults to read — the worker's +# pre-flight permission gate declines their commands (issue #13). +COLLABORATOR_PERMISSIONS = { + "octocat": "admin", + "coven-cody[bot]": "write", +} +DEFAULT_PERMISSION = "read" + +BRANCH_SHA = "a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6e7f8a9b0" + + +class World: + """All mutable stub state, guarded by one lock.""" + + def __init__(self): + self.lock = threading.Lock() + self.tokens = {} # token value -> role + self.token_seq = 0 + self.check_runs = {} # id -> dict + self.check_seq = 1000 + self.comments = [] # [{id, issue_number, body, user, edits}] + self.comment_seq = 5000 + self.pulls = {} # number -> dict + self.pull_seq = 100 + self.audit = [] # [{seq, at, actor, action}] + self.audit_seq = 0 + + def record(self, actor, action): + self.audit_seq += 1 + self.audit.append( + { + "seq": self.audit_seq, + "at": time.strftime("%H:%M:%S"), + "actor": actor, + "action": action, + } + ) + + def state(self): + return { + "tokens_minted": [ + {"role": role} for _, role in sorted(self.tokens.items()) + ], + "check_runs": list(self.check_runs.values()), + "comments": self.comments, + "pulls": list(self.pulls.values()), + "audit_len": len(self.audit), + } + + +WORLD = World() + + +def token_role(permissions): + """Names the scoped-token role from its permission set (issue #4).""" + if permissions.get("checks") == "write": + return "orchestration" + if permissions.get("pull_requests") == "write": + return "publication" + if permissions == {"contents": "write"}: + return "agent-git" + return "unknown" + + +class Handler(BaseHTTPRequestHandler): + protocol_version = "HTTP/1.1" + server_version = "github-stub/0.1" + + def log_message(self, fmt, *args): # quiet the default access log + pass + + def _json(self, status, payload): + body = json.dumps(payload).encode() + self.send_response(status) + self.send_header("Content-Type", "application/json") + self.send_header("Content-Length", str(len(body))) + self.end_headers() + self.wfile.write(body) + + def _read_body(self): + length = int(self.headers.get("Content-Length") or 0) + raw = self.rfile.read(length) if length else b"" + return json.loads(raw) if raw else {} + + def _actor(self): + auth = self.headers.get("Authorization", "") + token = auth.removeprefix("Bearer ").strip() + return WORLD.tokens.get(token, "app-jwt") + + # ── Routing ────────────────────────────────────────────────────────── + + def do_GET(self): + path = self.path.split("?")[0] + with WORLD.lock: + if path == "/_demo/state": + return self._json(200, WORLD.state()) + if path == "/_demo/audit": + return self._json(200, WORLD.audit) + + m = re.fullmatch(r"/repos/([^/]+)/([^/]+)", path) + if m: + WORLD.record(self._actor(), f"read repo metadata {m[1]}/{m[2]}") + return self._json(200, {"default_branch": "main"}) + + m = re.fullmatch(r"/repos/[^/]+/[^/]+/branches/([^/]+)", path) + if m: + WORLD.record(self._actor(), f"resolve branch '{m[1]}' head SHA") + return self._json(200, {"commit": {"sha": BRANCH_SHA}}) + + m = re.fullmatch( + r"/repos/[^/]+/[^/]+/collaborators/([^/]+)/permission", path + ) + if m: + perm = COLLABORATOR_PERMISSIONS.get(m[1], DEFAULT_PERMISSION) + WORLD.record( + self._actor(), + f"permission check for @{m[1]} -> {perm}", + ) + return self._json(200, {"permission": perm}) + + m = re.fullmatch(r"/repos/[^/]+/[^/]+/issues/(\d+)/comments", path) + if m: + number = int(m[1]) + items = [ + {"id": c["id"], "body": c["body"], "user": {"login": c["user"]}} + for c in WORLD.comments + if c["issue_number"] == number + ] + WORLD.record( + self._actor(), + f"list #{number} comments ({len(items)} found)", + ) + return self._json(200, items) + + self._json(404, {"message": f"stub: no route for GET {path}"}) + + def do_POST(self): + path = self.path.split("?")[0] + body = self._read_body() + with WORLD.lock: + m = re.fullmatch(r"/app/installations/(\d+)/access_tokens", path) + if m: + role = token_role(body.get("permissions", {})) + WORLD.token_seq += 1 + token = f"demo-token-{role}-{WORLD.token_seq}" + WORLD.tokens[token] = role + repos = ",".join(body.get("repositories", [])) + WORLD.record( + "app-jwt", + f"mint {role} token (repo-scoped: {repos})", + ) + return self._json(201, {"token": token}) + + m = re.fullmatch(r"/repos/([^/]+)/([^/]+)/check-runs", path) + if m: + WORLD.check_seq += 1 + run = { + "id": WORLD.check_seq, + "name": body.get("name", ""), + "head_sha": body.get("head_sha", ""), + "status": body.get("status", "queued"), + "conclusion": None, + "title": None, + "summary": None, + "details_url": body.get("details_url"), + } + WORLD.check_runs[run["id"]] = run + WORLD.record( + self._actor(), + f"create check run {run['id']} '{run['name']}' " + f"on {run['head_sha'][:12]} (queued)", + ) + return self._json(201, {"id": run["id"]}) + + m = re.fullmatch(r"/repos/[^/]+/[^/]+/issues/(\d+)/comments", path) + if m: + WORLD.comment_seq += 1 + comment = { + "id": WORLD.comment_seq, + "issue_number": int(m[1]), + "body": body.get("body", ""), + "user": "coven-cody[bot]", + "edits": 0, + } + WORLD.comments.append(comment) + WORLD.record( + self._actor(), + f"post comment {comment['id']} on #{m[1]}", + ) + return self._json(201, {"id": comment["id"]}) + + m = re.fullmatch(r"/repos/([^/]+)/([^/]+)/pulls", path) + if m: + WORLD.pull_seq += 1 + pull = { + "number": WORLD.pull_seq, + "title": body.get("title", ""), + "body": body.get("body", ""), + "head": body.get("head", ""), + "base": body.get("base", ""), + "draft": bool(body.get("draft")), + } + WORLD.pulls[pull["number"]] = pull + WORLD.record( + self._actor(), + f"open draft PR #{pull['number']} " + f"({pull['head']} -> {pull['base']})", + ) + return self._json(201, {"number": pull["number"]}) + + self._json(404, {"message": f"stub: no route for POST {path}"}) + + def do_PATCH(self): + path = self.path.split("?")[0] + body = self._read_body() + with WORLD.lock: + m = re.fullmatch(r"/repos/[^/]+/[^/]+/check-runs/(\d+)", path) + if m: + run = WORLD.check_runs.get(int(m[1])) + if not run: + return self._json(404, {"message": "no such check run"}) + run["status"] = body.get("status", run["status"]) + if "conclusion" in body: + run["conclusion"] = body["conclusion"] + output = body.get("output") or {} + run["title"] = output.get("title", run["title"]) + run["summary"] = output.get("summary", run["summary"]) + detail = run["conclusion"] or run["status"] + WORLD.record( + self._actor(), + f"check run {run['id']} -> {detail} ('{run['title']}')", + ) + return self._json(200, {"id": run["id"]}) + + m = re.fullmatch(r"/repos/[^/]+/[^/]+/issues/comments/(\d+)", path) + if m: + cid = int(m[1]) + for comment in WORLD.comments: + if comment["id"] == cid: + comment["body"] = body.get("body", comment["body"]) + comment["edits"] += 1 + first_line = next( + ( + line + for line in comment["body"].splitlines() + if line and not line.startswith("