Merge pull request #121 from clkao/spacedock-ensign/cherry-pick-claude-team-narrowing

Narrow claude-team find_subagent_jsonl scan to team session dir

Author: clkao

docs/plans/cherry-pick-claude-team-narrowing.md

@@ -161,3 +161,36 @@ land the cherry-pick.
 ### Summary
 
 Confirmed `b09051f4` is a clean, single-file cherry-pick onto current `main`. Expanded the entity body with a problem statement, approach with pre/post code shape, five acceptance criteria (each with a test column), and a proportional test plan (static suite required, cheap unit test with four cases flagged optional, no live CI). The independent reviewer's stderr-warning question is captured as informational only — the warning fires only on narrowed-succeeded-but-empty, not on every normal run. Scope kept tight: no prose changes, no test-predicate changes, no warning-tuning. Implementation stage is unblocked.
+
+## Stage Report (implementation)
+
+1. **Stay inside worktree `/Users/clkao/git/spacedock/.worktrees/spacedock-ensign-cherry-pick-claude-team-narrowing` on branch `spacedock-ensign/cherry-pick-claude-team-narrowing`.** — **DONE.** All commands executed inside the worktree; `git branch --show-current` confirmed the expected branch before and after cherry-pick.
+2. **Read the entity body's approach + acceptance criteria + test plan sections.** — **DONE.** Scope confirmed: cherry-pick `b09051f4`, verify single-file diff, run static suite; optional unit test evaluated below.
+3. **Cherry-pick commit `b09051f4` from `origin/spacedock-ensign/diagnose-opus-4-7-fo-regression`.** — **DONE.** Fetched origin ref, ran `git cherry-pick b09051f4`; applied cleanly with no conflicts. New commit on branch: `3d5b8f15`.
+4. **Verify the cherry-picked changes touch ONLY `skills/commission/bin/claude-team`.** — **DONE.** `git show HEAD --stat` reports `1 file changed, 63 insertions(+), 10 deletions(-)` against `skills/commission/bin/claude-team`. No other paths touched.
+5. **Run the static suite.** — **DONE.** Initial run after cherry-pick: `unset CLAUDECODE && make test-static` → `426 passed, 22 deselected, 10 subtests passed in 20.32s`. Re-run after the SyntaxWarning fix (item 9): `426 passed ... in 19.85s`. No new failures vs `main`.
+6. **Evaluate optional unit test for narrowed helper.** — **SKIPPED.** Rationale: the entity's test plan explicitly flags this as optional and "not required to land the cherry-pick"; the dispatch scope says "Do NOT modify tests unless the entity's approach specifies a unit test addition" — the approach section does not mandate it. Also, adding the test would expand scope beyond a pure cherry-pick and introduce surface area (fixture scaffolding, `os.path.expanduser` monkey-patching) that the dispatch explicitly warns against. Acceptance criterion #5 (fallback path) is indirectly exercised by the static suite's import+parse path and the helper's contract (unchanged return shape) matches the pre-commit behavior; follow-up task can add the four-case unit test if operational data motivates it.
+7. **Commit changes on the worktree branch.** — **DONE.** Cherry-pick produced commit `3d5b8f15`; SyntaxWarning fix (item 9) produced `5c278b68`; stage report commit follows this write.
+8. **Write this Stage Report (implementation) section.** — **DONE** (this section).
+9. **Fix SyntaxWarning on `claude-team` line 46 (captain scope expansion).** — **DONE.** Converted the `extract_stage_subsection` docstring to a raw string (`r"""..."""`) so the literal `` \` `` inside the example heading stops being interpreted as an invalid escape sequence. Verified: `python3 -W all skills/commission/bin/claude-team list-standing --workflow-dir /Users/clkao/git/spacedock/docs/plans` and `python3 skills/commission/bin/claude-team --help` — both produce clean stderr, no SyntaxWarning. Single-line diff (+1 / −1). Static suite re-run: 426 passed.
+
+### Summary
+
+Cherry-picked `b09051f4` onto `spacedock-ensign/cherry-pick-claude-team-narrowing` with no conflicts (commit `3d5b8f15`). Verified single-file scope — `skills/commission/bin/claude-team` (+63/−10). Static suite green: 426 passed, 22 deselected, 0 failures. Skipped the optional four-case unit test with rationale (entity flags optional, dispatch scope excludes test additions not mandated in approach). Passenger fix (commit `5c278b68`): converted `extract_stage_subsection` docstring to a raw string, silencing the pre-existing SyntaxWarning on line 46 (verified via `list-standing` and `--help` invocations). Ready for validation.
+
+## Stage Report (validation)
+
+1. **Read both prior stage reports in the entity body.** — **[x] DONE.** Read ideation and implementation reports. Noted the audit gap: the implementation report was written before commit `5c278b68` was appended mid-stage via a captain-scope expansion, so that commit is summarized as an addendum (item 9) rather than integrated into the top-level summary. Not a blocker; dispatch flagged this explicitly.
+2. **Audit gap noted.** — **[x] DONE.** Two commits landed on the worktree branch beyond the entering-state base: `3d5b8f15` (cherry-pick) and `5c278b68` (SyntaxWarning fix). Plus `4d9083f8` + `d7b9b63e` stage-report commits. The SyntaxWarning fix was out-of-scope for the original ideation but in-scope per captain approval mid-implementation; audit trail captured in item 9 of the implementation report.
+3. **Verify AC-1 (clean cherry-pick).** — **[x] DONE.** `git log --oneline -5` shows `3d5b8f15 fix: #182 narrow find_subagent_jsonl scan to one team's leadSessionId` on the worktree branch. `git show 3d5b8f15 --stat` confirms `1 file changed, 63 insertions(+), 10 deletions(-)` against `skills/commission/bin/claude-team` — single-file scope matches the ideation's claim exactly.
+4. **Verify AC-2 (semantic match with b09051f4).** — **[x] DONE.** Read lines 354-438 of `skills/commission/bin/claude-team` in the worktree. Structure matches the ideation's pre/post description: `find_subagent_jsonl` (lines 354-394) orchestrates narrowed-first-then-fallback; `_narrowed_subagent_patterns` (397-421) returns a single-element glob list scoped to `leadSessionId` or `None`; `_scan_subagent_meta` (424-438) is the extracted scan loop. Stderr warning fires only on the narrowed-succeeded-but-empty branch (lines 380-385).
+5. **Verify AC-3 (static suite).** — **[x] DONE.** `unset CLAUDECODE && make test-static` → `426 passed, 22 deselected, 10 subtests passed in 20.18s`. Meets the ≥ 426 threshold. No new failures vs main.
+6. **Verify AC-4 (empirical narrowing on this machine).** — **[x] DONE.** Broad scan haystack: 4863 `agent-*.meta.json` files across both broad patterns (0 at depth-1, 4863 at depth-2). 53 team configs present under `~/.claude/teams/*/config.json`. Sampled three configs: `test-project-rejection-pipeline` (narrow=0), `spacedock-plans-20260417-2342-f26a6ff0` (narrow=16), `sparkling-rolling-adleman` (narrow=9). Reduction: **~300–500× fewer meta-file opens** per lookup when narrowing applies. Matches the commit-message claim (empirically narrows ~4816 → ~9). `./skills/commission/bin/claude-team context-budget --name comm-officer` returned clean JSON (`resident_tokens: 42690`, `reuse_ok: true`, exit 0) — narrowed path exercised successfully via this workflow's live team config (`spacedock-plans-20260417-2342-f26a6ff0`). `list-standing --workflow-dir .../docs/plans` also ran clean.
+7. **Verify AC-5 (fallback path exists).** — **[x] DONE.** Code inspection (lines 387-388) confirms: when `_narrowed_subagent_patterns` returns `None`, control flows straight to `_scan_subagent_meta(broad_patterns, name)` with no warning. When narrowed returns a pattern but `_scan_subagent_meta` is empty, stderr warns then falls back (lines 380-386). Return contract (`str | None`) is unchanged from the pre-commit signature.
+8. **Verify SyntaxWarning fix (passenger).** — **[x] DONE.** Worktree binary: `python3 -W all ./skills/commission/bin/claude-team list-standing --workflow-dir /Users/clkao/git/spacedock/docs/plans 2>&1 | grep -i "syntaxwarning"` returns empty — **no warning**. Main-branch binary (baseline): `python3 -W all /Users/clkao/git/spacedock/skills/commission/bin/claude-team --help 2>&1 | grep -i "syntaxwarning"` returns `/Users/clkao/git/spacedock/skills/commission/bin/claude-team:46: SyntaxWarning: "\`" is an invalid escape sequence.` — warning present on main, absent on worktree. Fix is scoped correctly.
+9. **Recommend PASSED or REJECTED.** — **[x] DONE. Recommendation: PASSED.** All five acceptance criteria met with evidence. SyntaxWarning passenger fix verified scoped correctly. Static suite green. No regressions. Single-file cherry-pick surface area matches ideation.
+10. **Write this Stage Report (validation) section.** — **[x] DONE** (this section).
+
+### Summary
+
+Validation PASSED. Cherry-pick `3d5b8f15` applies cleanly as a single-file change (+63/−10 on `skills/commission/bin/claude-team`) and semantically matches `b09051f4` (narrowed-first-then-broad-fallback with stderr warning only on narrowed-succeeded-but-empty). Static suite: 426 passed, 0 regressions. Empirical narrowing confirmed on this machine: 4863-file broad haystack reduced to 9-16 files for active teams (~300-500× reduction), matching the commit-message claim. Fallback path preserved (code + live `context-budget --name comm-officer` invocation confirm). Passenger SyntaxWarning fix (commit `5c278b68`) verified scoped correctly — warning absent on worktree binary, present on main-branch binary. Audit gap noted: implementation report written before the SyntaxWarning commit landed, but item 9 of that report documents the addendum honestly. Ready for merge gate.

skills/commission/bin/claude-team

@@ -41,7 +41,7 @@ MODEL_ENUM_LIST = 'must be one of: sonnet, opus, haiku'
 
 
 def extract_stage_subsection(readme_path, stage_name):
-    """Extract the full ### {stage_name} subsection from a workflow README.
+    r"""Extract the full ### {stage_name} subsection from a workflow README.
 
     Accepts both backtick-quoted (`### \`work\``) and bare (`### work`) headings
     so fixtures and hand-authored workflows both parse cleanly.
@@ -354,17 +354,75 @@ THRESHOLD_PCT = 60
 def find_subagent_jsonl(name: str) -> str | None:
     """Find the most recently modified subagent jsonl matching the given name.
 
-    Scans ~/.claude/projects/*/subagents/agent-*.meta.json for agentType matching name.
-    Also scans with session subdirectory: ~/.claude/projects/*/*/subagents/agent-*.meta.json.
+    Looks up the team config containing a member named `name`, reads its
+    `leadSessionId`, and globs only that session's subagents directory:
+    ~/.claude/projects/*/{leadSessionId}/subagents/agent-*.meta.json. On a
+    populated machine this narrows the haystack from thousands of files
+    (every project's subagents) to dozens (one session's subagents).
+
+    Falls back to the broad scan if no team config is found, no
+    leadSessionId is set, or the narrowed scan returns nothing — the broad
+    scan stays correct, just slow. A warning is emitted to stderr in the
+    fallback case.
+
+    Scans agent-*.meta.json for agentType matching name.
     Returns the sibling .jsonl path, or None.
     """
     home = os.path.expanduser("~")
-    patterns = [
+    broad_patterns = [
         os.path.join(home, ".claude", "projects", "*", "subagents", "agent-*.meta.json"),
         os.path.join(home, ".claude", "projects", "*", "*", "subagents", "agent-*.meta.json"),
     ]
 
-    matches = []
+    narrowed = _narrowed_subagent_patterns(home, name)
+    if narrowed is not None:
+        matches = _scan_subagent_meta(narrowed, name)
+        if not matches:
+            print(
+                f"warning: narrowed subagent scan for {name!r} found nothing; "
+                f"falling back to broad ~/.claude/projects scan",
+                file=sys.stderr,
+            )
+            matches = _scan_subagent_meta(broad_patterns, name)
+    else:
+        matches = _scan_subagent_meta(broad_patterns, name)
+
+    if not matches:
+        return None
+
+    matches.sort(key=lambda x: x[0], reverse=True)
+    return matches[0][1]
+
+
+def _narrowed_subagent_patterns(home: str, name: str) -> list[str] | None:
+    """Return narrowed glob patterns for a subagent jsonl scoped to one team session.
+
+    Returns None when no team config containing `name` declares a leadSessionId.
+    """
+    teams_pattern = os.path.join(home, ".claude", "teams", "*", "config.json")
+    for config_path in glob.glob(teams_pattern):
+        try:
+            with open(config_path, "r") as f:
+                config = json.load(f)
+        except (json.JSONDecodeError, OSError):
+            continue
+        member_names = {m.get("name") for m in config.get("members", []) if isinstance(m, dict)}
+        if name not in member_names:
+            continue
+        lead_session_id = config.get("leadSessionId")
+        if not lead_session_id:
+            continue
+        return [
+            os.path.join(
+                home, ".claude", "projects", "*", lead_session_id, "subagents",
+                "agent-*.meta.json",
+            ),
+        ]
+    return None
+
+
+def _scan_subagent_meta(patterns: list[str], name: str) -> list[tuple[float, str]]:
+    matches: list[tuple[float, str]] = []
     for pattern in patterns:
         for meta_path in glob.glob(pattern):
             try:
@@ -377,12 +435,7 @@ def find_subagent_jsonl(name: str) -> str | None:
                         matches.append((mtime, jsonl_path))
             except (json.JSONDecodeError, OSError):
                 continue
-
-    if not matches:
-        return None
-
-    matches.sort(key=lambda x: x[0], reverse=True)
-    return matches[0][1]
+    return matches
 
 
 def extract_resident_tokens(jsonl_path: str) -> int | None: