Skip to content

Commit f344b23

Browse files
authored
Merge pull request #26 from anombyte93/feat/marketplace-watcher
feat(tournament): independent out-of-band re-execution watcher gating real slashing
2 parents 4969a13 + 956fd43 commit f344b23

7 files changed

Lines changed: 1169 additions & 6 deletions

File tree

CHANGELOG.md

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,25 @@ All notable changes to this project are documented here. Format based on
44
[Keep a Changelog](https://keepachangelog.com/en/1.1.0/); this project adheres to
55
[Semantic Versioning](https://semver.org/spec/v2.0.0.html).
66

7+
## [Unreleased]
8+
9+
### Added
10+
- **Independent out-of-band re-execution watcher** (`prd_taskmaster/tournament/watcher.py`;
11+
`watcher-run` / `watcher-status` CLI) — the precondition for ever enabling real
12+
(`--enforce-slash`) forfeiture. It re-adjudicates settled tournament submissions from
13+
**primary evidence** (the claimed commit + the CDD card) without trusting the recorded
14+
verdict: it re-runs the oracle gate, re-derives `sha256(diff base..HEAD)` to catch diff-copy
15+
tamper independently of the live collector, and accumulates a concordance ledger over real
16+
*slash decisions*. A `permit_enforce_slash` gate is **fail-closed**: real slashing is permitted
17+
only when every to-be-slashed submission is independently confirmed, there is **no discrepancy
18+
or abstain anywhere in the job**, and the watcher's historical concordance clears a threshold
19+
over a minimum number of prior decisions. Inability to verify (oracle could not run, no
20+
worktree, failed hash recompute) **abstains** — it is never counted as grounds to slash.
21+
- **Engine-enforced shadow-until-permitted**`run_tournament` now consults the watcher before
22+
any real `--enforce-slash` settle and **downgrades to shadow** unless the watcher permits the
23+
job (fail-closed on any watcher error). Real AtlasCoin is never burned without an independent
24+
positive confirmation; the shadow-slash default path is unchanged.
25+
726
## [5.3.0] — 2026-06-17
827

928
The "unfakable done" release: a task ships only when its test genuinely re-ran green AND

CLAUDE.md

Lines changed: 14 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -53,9 +53,16 @@ a UCB reputation store routes the next job to the cheapest proven-capable execut
5353
- `collect.py` is the security core — **commit-reveal** (`commit_hash = sha256(diff base..HEAD)`)
5454
defeats the diff-copy attack. `antisybil.py` enforces per-job / per-operator economic caps.
5555
- `goose_backend.py` is the cheap-API racer (one OpenRouter model via `goose run`).
56-
- **Invariants (enforced + tested — keep them):** **shadow-slash only** until an independent
57-
out-of-band watcher exists (proven cheating is logged, no AtlasCoin burned); honest losers
58-
are **always refunded**; **AtlasCoin is conserved** (no mint/burn, even under account aliasing).
56+
- `watcher.py` (`watcher-run` / `watcher-status`) is the **independent out-of-band re-execution
57+
watcher** — the precondition for real slashing. It re-adjudicates settled submissions from
58+
primary evidence (re-runs the oracle, independently re-derives `sha256(diff base..HEAD)` to
59+
catch diff-copy), abstains when it cannot verify (never a false confirm), and **engine-gates**
60+
real forfeiture: `run_tournament` downgrades `--enforce-slash` to shadow unless the fail-closed
61+
`permit_enforce_slash` confirms the whole job behind a concordance track record.
62+
- **Invariants (enforced + tested — keep them):** **shadow-slash by default**; real slashing only
63+
behind the watcher's fail-closed permit (no AtlasCoin burned without an independent positive
64+
confirmation); honest losers are **always refunded**; **AtlasCoin is conserved** (no mint/burn,
65+
even under account aliasing).
5966

6067
## The spine (`atlas-protocol`, separate repo)
6168

@@ -94,7 +101,8 @@ Key modules: `cli.py` (dispatch) · `backend.py` (5-op backend protocol) · `val
94101
`tasks.py` (graded PRD + task graph) · `task_state.py` (atomic next/claim/set-status) ·
95102
`reachability.py` (Gate 6) · `oracle_bridge.py` + `shipcheck.py` (Gate 5 wiring) · `economy.py`
96103
+ `reputation.py` (cost ledger + UCB routing) · `fleet.py` / `parallel.py` (fleet waves) ·
97-
`tournament/` (the settled marketplace). The binding ship gate is `skel/ship-check.py`; the
104+
`tournament/` (the settled marketplace; `tournament/watcher.py` is the independent re-execution
105+
watcher that engine-gates real slashing). The binding ship gate is `skel/ship-check.py`; the
98106
skill/slash commands live in `skills/` (atlas, go, generate, execute-task, execute-fleet, …) and
99107
ship inside the package. `SKILL.md` is the normative behavior spec. Tests split into
100108
`tests/core` + `tests/plugin` (stdlib-only), `tests/mcp` (needs the mcp package), and
@@ -126,6 +134,8 @@ python3 script.py engine-preflight # one-call environment + back
126134
python3 script.py validate-tasks <tasks.json> # gate before any tasks.json write
127135
python3 script.py reachability-sweep # Gate 6 — orphan-module detection
128136
python3 script.py tournament-run # settled marketplace: spawn→…→reputation
137+
python3 script.py watcher-run --job <dir> --card <card> --task <id> --base-ref <sha> # out-of-band re-adjudication
138+
python3 script.py watcher-status # watcher concordance + whether real slashing is permitted yet
129139
python3 script.py economy-report # cost ledger from .atlas-ai/telemetry.jsonl
130140
```
131141

prd_taskmaster/cli.py

Lines changed: 28 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,12 @@
2626
from prd_taskmaster import fleet, parallel, task_state
2727
from prd_taskmaster.lib import _detect_taskmaster_method
2828
from prd_taskmaster.reachability_cmd import cmd_reachability_sweep
29-
from prd_taskmaster.tournament.cmd import cmd_tournament_run, cmd_tournament_status
29+
from prd_taskmaster.tournament.cmd import (
30+
cmd_tournament_run,
31+
cmd_tournament_status,
32+
cmd_watcher_run,
33+
cmd_watcher_status,
34+
)
3035

3136

3237
def _backend_source() -> str:
@@ -419,6 +424,26 @@ def build_parser() -> argparse.ArgumentParser:
419424
p.add_argument("--reputation-path", default=None, help="Path to reputation.jsonl (default: .atlas-ai/reputation.jsonl)")
420425
p.add_argument("--operators-path", default=None, help="Path to operators.json (default: .atlas-ai/tournament/operators.json)")
421426

427+
# watcher-run — independent out-of-band re-adjudication of a settled job
428+
p = sub.add_parser(
429+
"watcher-run",
430+
help="Re-adjudicate a settled tournament job out-of-band; append concordance; report the real-slash permit",
431+
)
432+
p.add_argument("--job", required=True, help="Path to the settled job dir (contains submissions.json)")
433+
p.add_argument("--card", required=True, help="Path to the CDD card JSON")
434+
p.add_argument("--task", required=True, help="Task id (e.g. 7 or 1.2)")
435+
p.add_argument("--base-ref", required=True, help="Fork-point git SHA the diff range is measured from")
436+
p.add_argument("--repo-root", default=".", help="Repo containing the racer commits (default: .)")
437+
p.add_argument("--held-root", default=None, help="Held root for the oracle gate (default: .atlas-ai/cdd)")
438+
p.add_argument("--ledger-path", default=None, help="Watcher ledger path (default: .atlas-ai/tournament/watcher.jsonl)")
439+
440+
# watcher-status — report watcher concordance + real-slash readiness
441+
p = sub.add_parser(
442+
"watcher-status",
443+
help="Show the watcher's historical concordance and whether real slashing is permitted yet",
444+
)
445+
p.add_argument("--ledger-path", default=None, help="Watcher ledger path (default: .atlas-ai/tournament/watcher.jsonl)")
446+
422447
# status — render progress panels
423448
p = sub.add_parser("status", help="Render Atlas progress panels for the current phase")
424449
p.add_argument("--phase", default=None, help="Render a specific phase instead of the current one")
@@ -476,6 +501,8 @@ def cmd_status(args) -> None:
476501
"reachability-sweep": cmd_reachability_sweep,
477502
"tournament-run": cmd_tournament_run,
478503
"tournament-status": cmd_tournament_status,
504+
"watcher-run": cmd_watcher_run,
505+
"watcher-status": cmd_watcher_status,
479506
"economy-report": cmd_economy_report,
480507
"context-pack": cmd_context_pack,
481508
"feedback-add": cmd_feedback_add,

prd_taskmaster/tournament/cmd.py

Lines changed: 110 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,7 @@
3333

3434
import argparse
3535
import json
36+
import subprocess
3637
import sys
3738
from pathlib import Path
3839
from typing import Any, Callable, Optional
@@ -47,9 +48,28 @@
4748
FakeClock,
4849
)
4950
from prd_taskmaster.tournament.adjudicate import adjudicate_job, settle_job
51+
from prd_taskmaster.tournament import watcher as _watcher
5052
from prd_taskmaster.reputation import record_tournament, summarize_reputation, _winner_id
5153

5254

55+
def _resolve_repo_root(cwd: "str | None" = None, default: str = ".") -> str:
56+
"""Resolve the git repo root for the watcher's re-execution worktrees.
57+
58+
Falls back to *default* (never raises) when not in a git repo, so the
59+
watcher gate degrades gracefully rather than crashing the settle path.
60+
"""
61+
try:
62+
proc = subprocess.run(
63+
["git", "rev-parse", "--show-toplevel"],
64+
cwd=cwd, capture_output=True, text=True, timeout=10,
65+
)
66+
if proc.returncode == 0 and proc.stdout.strip():
67+
return proc.stdout.strip()
68+
except Exception: # noqa: BLE001
69+
pass
70+
return default
71+
72+
5373
# ─── Core orchestration function ─────────────────────────────────────────────
5474

5575

@@ -74,6 +94,10 @@ def run_tournament(
7494
now: str,
7595
window_s: float = 120.0,
7696
enforce_slash: bool = False,
97+
repo_root: str = ".",
98+
watcher_ledger_path: "str | Path | None" = None,
99+
_re_adjudicate: "Optional[Callable[..., dict]]" = None,
100+
_permit: "Optional[Callable[..., dict]]" = None,
77101
_spawn_fn: "Callable[[Any], dict]" = default_launcher_adapter,
78102
_inbox_read: "Callable[..., list[dict]]" = default_inbox_adapter,
79103
_dispatch_reveal: "Callable[..., Optional[dict]]" = default_reveal_adapter,
@@ -290,8 +314,38 @@ def run_tournament(
290314
job_poster=job_poster,
291315
)
292316

317+
# ── Step 5b: Watcher gate on REAL slashing (engine-enforced) ─────
318+
# Shadow-slash only until the independent out-of-band watcher PERMITS
319+
# real forfeiture for THIS job. Fail-closed: a watcher error, a failed
320+
# re-adjudication, or a non-permit downgrades enforce_slash to shadow
321+
# so no AtlasCoin is ever burned without an independent confirmation.
322+
effective_enforce = enforce_slash
323+
if enforce_slash:
324+
_radj = _re_adjudicate or _watcher.re_adjudicate_job
325+
_perm = _permit or _watcher.permit_enforce_slash
326+
w_ledger = (
327+
Path(watcher_ledger_path) if watcher_ledger_path
328+
else Path(".atlas-ai/tournament/watcher.jsonl")
329+
)
330+
try:
331+
wrec = _radj(
332+
job_dir=job_dir, repo_root=repo_root, card_path=card_path,
333+
held_root=held_root, task_id=task_id, start_commit=base_ref,
334+
base_ref=base_ref, now=now, ledger_path=w_ledger,
335+
)
336+
if not wrec.get("ok"):
337+
permit = {"permitted": False, "reason": "watcher re-adjudication failed"}
338+
else:
339+
permit = _perm(wrec, ledger_path=w_ledger, current_job_id=job_id)
340+
except Exception as exc: # noqa: BLE001 — fail-closed → shadow
341+
permit = {"permitted": False, "reason": f"watcher error: {exc}"}
342+
summary["watcher_permit"] = permit
343+
if not permit.get("permitted"):
344+
effective_enforce = False
345+
summary["enforce_slash_downgraded"] = True
346+
293347
# ── Step 6: Settle (FAIL-CLOSED on ok:false) ─────────────────────
294-
settle_env = _settle_fn(job_dir=job_dir, enforce_slash=enforce_slash)
348+
settle_env = _settle_fn(job_dir=job_dir, enforce_slash=effective_enforce)
295349

296350
settled_ok = settle_env.get("ok") is True
297351
summary["settled_ok"] = settled_ok
@@ -415,6 +469,7 @@ def cmd_tournament_run(args: argparse.Namespace) -> None:
415469
now=now,
416470
window_s=float(getattr(args, "window", 120.0)),
417471
enforce_slash=bool(getattr(args, "enforce_slash", False)),
472+
repo_root=_resolve_repo_root(),
418473
# Real adapters must be wired by the orchestrator skill; the defaults
419474
# raise RuntimeError with guidance if called directly.
420475
_spawn_fn=default_launcher_adapter,
@@ -426,6 +481,60 @@ def cmd_tournament_run(args: argparse.Namespace) -> None:
426481
_emit({"ok": False, "error": str(exc)})
427482

428483

484+
def cmd_watcher_run(args: argparse.Namespace) -> None:
485+
"""CLI handler for ``watcher-run``.
486+
487+
Re-adjudicates a settled job out-of-band from primary evidence, appends a
488+
concordance row to the watcher ledger, and reports the fail-closed
489+
real-slash permit for the job. FAIL-CLOSED: a missing/unreadable job → ok:false.
490+
"""
491+
job_dir = Path(args.job)
492+
repo_root = getattr(args, "repo_root", None) or "."
493+
card_path = Path(args.card)
494+
held_root = Path(getattr(args, "held_root", None) or ".atlas-ai/cdd")
495+
ledger_path = Path(getattr(args, "ledger_path", None) or ".atlas-ai/tournament/watcher.jsonl")
496+
497+
import datetime
498+
now = datetime.datetime.now(datetime.timezone.utc).isoformat()
499+
500+
try:
501+
rec = _watcher.re_adjudicate_job(
502+
job_dir=job_dir,
503+
repo_root=repo_root,
504+
card_path=card_path,
505+
held_root=held_root,
506+
task_id=args.task,
507+
start_commit=args.base_ref,
508+
base_ref=args.base_ref,
509+
now=now,
510+
ledger_path=ledger_path,
511+
)
512+
except Exception as exc: # noqa: BLE001
513+
_emit({"ok": False, "error": str(exc)})
514+
return
515+
516+
if not rec.get("ok"):
517+
_emit({"ok": False, **rec})
518+
return
519+
520+
# Exclude the in-flight job (just appended to the ledger) from its own gate.
521+
permit = _watcher.permit_enforce_slash(
522+
rec, ledger_path=ledger_path, current_job_id=rec.get("job_id")
523+
)
524+
_emit({"ok": True, **rec, "permit": permit})
525+
526+
527+
def cmd_watcher_status(args: argparse.Namespace) -> None:
528+
"""CLI handler for ``watcher-status``.
529+
530+
Reports the watcher's historical concordance and whether the track record
531+
would clear the real-slash gate (``real_slash_ready``). A view command.
532+
"""
533+
ledger_path = Path(getattr(args, "ledger_path", None) or ".atlas-ai/tournament/watcher.jsonl")
534+
summ = _watcher.concordance_summary(ledger_path)
535+
_emit({"ok": True, **summ})
536+
537+
429538
def cmd_tournament_status(args: argparse.Namespace) -> None:
430539
"""CLI handler for ``tournament-status``.
431540

0 commit comments

Comments
 (0)