Skip to content

Commit 59fe23d

Browse files
author
bgagent
committed
feat(registry): orchestrator + agent + MCP server E2E (#246)
PR 2 of 3 for the central agent asset registry (ADR-018). Wires the registry (PR 1) end-to-end for one asset kind: a blueprint pins an MCP server, the orchestrator resolves it at the create-task boundary, stamps the resolved {kind,id,version} on the TaskRecord, and the agent merges the resolved server config into .mcp.json alongside the channel MCP. - Blueprint: assets.mcpServers prop -> RepoConfig mcp_servers column, with a synth-time RegistryRefValidation (rejects floating/malformed refs). Grammar extracted to a dependency-free registry-ref.ts so the construct layer can validate without pulling aws-sdk into the synth graph. - Orchestrator: resolveRegistryAssetsForTask resolves the blueprint's pins before hydration (fail-fast/fail-closed), stamps resolved_assets, threads the bundle into the agent payload. resolved_assets added to TaskRecord/TaskDetail/ TaskSummary (+ CLI mirror). - Agent: registry_loader.py applies the resolved bundle — apply_mcp_assets merges server_config into .mcp.json; cedar/skill loaders stubbed for PR 3. resolved_assets threaded server.py -> pipeline.py -> TaskConfig. - CLI: bgagent registry publish/resolve/list/show. Deploy-hardening fixes found validating against a live stack: - Create RegistryPublisher/RegistryApprover Cognito groups (were referenced by the publish handler but never created). Bootstrap action-map + policy + DEPLOYMENT_ROLES.md updated for cognito-idp group actions. - CLI publish flag --asset-version (--version collided with commander's global). - memorySize=256 on the four registry Lambdas (128MB default OOM'd on init). - mcp_server publish accepts inline descriptor server_config in lieu of an artifact. - Orchestrator Lambda gets REGISTRY_ASSETS_TABLE_NAME env + read grant (resolve runs there during hydration). Validated live (backgroundagent-pr246): resolved_assets correctly stamped on a real TaskRecord. Task itself fails on a pre-existing 'claude' Exec-format image bug, unrelated to the registry. Registry MCP secret-injection (${VAR} from Secrets Manager) is a tracked follow-up, out of MVP scope per ADR-018.
1 parent 773764f commit 59fe23d

32 files changed

Lines changed: 1430 additions & 92 deletions

agent/src/models.py

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -221,6 +221,12 @@ class TaskConfig(BaseModel):
221221
trace: bool = False
222222
# Enriched mid-flight by pipeline.py:
223223
cedar_policies: list[str] = []
224+
# Registry (#246): the resolved asset bundle threaded from the orchestrator
225+
# payload — ``{mcp_servers, cedar_policy_modules, skills}`` each a list of
226+
# resolved-asset dicts. Applied by registry_loader.py at task start (MVP:
227+
# mcp_servers merged into .mcp.json; cedar/skills staged for PR 3). Empty
228+
# dict when the blueprint pinned no assets.
229+
resolved_assets: dict[str, list[dict]] = {}
224230
# Cedar HITL (§7.3, §10.2). Per-task approval defaults threaded
225231
# from the orchestrator payload; consumed by PolicyEngine at
226232
# construction so the engine seeds ApprovalAllowlist and adopts

agent/src/pipeline.py

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -40,6 +40,7 @@
4040
)
4141
from progress_writer import _ProgressWriter
4242
from prompt_builder import build_system_prompt, discover_project_config
43+
from registry_loader import apply_resolved_assets
4344
from shell import log, log_error_cw
4445
from telemetry import (
4546
_TrajectoryWriter,
@@ -616,6 +617,7 @@ def run_task(
616617
branch_name: str = "",
617618
pr_number: str = "",
618619
cedar_policies: list[str] | None = None,
620+
resolved_assets: dict[str, list[dict]] | None = None,
619621
approval_timeout_s: int | None = None,
620622
initial_approvals: list[str] | None = None,
621623
initial_approval_gate_count: int = 0,
@@ -670,6 +672,12 @@ def run_task(
670672
if cedar_policies:
671673
config.cedar_policies = cedar_policies
672674

675+
# Registry (#246): thread the resolved asset bundle onto config so the
676+
# loader (below, after channel MCP wiring) can apply it. Same post-build
677+
# injection pattern as cedar_policies.
678+
if resolved_assets:
679+
config.resolved_assets = resolved_assets
680+
673681
# Export session-tag values so tenant-data boto3 clients (DDB/S3) assume
674682
# the per-task SessionRole with {user_id, repo, task_id} tags. No-op when
675683
# AGENT_SESSION_ROLE_ARN is unset (local/dev/tests).
@@ -867,6 +875,12 @@ def _on_trace_truncated(max_bytes: int, first_dropped: int) -> None:
867875
resolve_jira_oauth_token(config.channel_metadata)
868876
configure_channel_mcp(setup.repo_dir, config.channel_source)
869877

878+
# Registry (#246): apply resolved assets AFTER the channel MCP so a
879+
# registry-published MCP server merges alongside (not over) the
880+
# channel entry in .mcp.json, and both are present before
881+
# discover_project_config scans. No-op when no assets were pinned.
882+
apply_resolved_assets(setup.repo_dir, config.resolved_assets)
883+
870884
# 👀 on the Linear issue — acknowledges the task is picked up.
871885
# No-op for non-Linear tasks. Best-effort; failures are logged
872886
# but do not block the pipeline. Capture the reaction id so we

agent/src/registry_loader.py

Lines changed: 163 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,163 @@
1+
"""Apply registry-resolved assets (#246) to the agent's runtime environment.
2+
3+
The orchestrator resolves a blueprint's ``registry://`` pins at the create-task
4+
boundary and threads a ``resolved_assets`` bundle into the invocation payload
5+
(see docs/design/REGISTRY.md §7/§8). This module is the agent-side consumer:
6+
it takes that bundle and applies each asset kind with the right mechanism.
7+
8+
MVP (this PR) wires ``mcp_server`` end-to-end — the resolved server config is
9+
merged into ``<repo_dir>/.mcp.json`` alongside whatever ``channel_mcp.py``
10+
wrote, so the Claude Agent SDK (``setting_sources=["project"]``) picks it up at
11+
session start. ``cedar_policy_module`` and ``skill`` are stubs here; PR 3 fills
12+
them in (Cedar text appended to the PolicyEngine set; skill prompt fragments
13+
into the SDK setting sources).
14+
15+
Design note — parity with channel_mcp.py: both write ``.mcp.json`` by
16+
read-merge-write on the ``mcpServers`` map, never clobbering other servers.
17+
Registry assets and the channel MCP can therefore coexist in one file.
18+
"""
19+
20+
from __future__ import annotations
21+
22+
import json
23+
import os
24+
from typing import Any
25+
26+
from shell import log
27+
28+
#: Key under which the resolved MCP server config lives in the descriptor
29+
#: (REGISTRY.md §3.3). The value is a ready-to-write ``mcpServers`` entry.
30+
_MCP_SERVER_CONFIG_KEY = "server_config"
31+
32+
33+
def _read_existing_mcp_config(path: str) -> dict[str, Any]:
34+
"""Return the parsed .mcp.json at ``path``, or an empty dict if absent/invalid.
35+
36+
Mirrors ``channel_mcp._read_existing_mcp_config`` — malformed JSON is logged
37+
and treated as absent rather than crashing the agent on a broken file.
38+
"""
39+
if not os.path.isfile(path):
40+
return {}
41+
try:
42+
with open(path, encoding="utf-8") as f:
43+
parsed = json.load(f)
44+
if isinstance(parsed, dict):
45+
return parsed
46+
log("WARN", f"Ignoring non-object .mcp.json at {path} (got {type(parsed).__name__})")
47+
except (OSError, json.JSONDecodeError) as e:
48+
log("WARN", f"Failed to read existing .mcp.json at {path}: {type(e).__name__}: {e}")
49+
return {}
50+
51+
52+
def _mcp_server_key(asset: dict[str, Any]) -> str:
53+
"""The ``mcpServers`` key an asset registers under.
54+
55+
Prefer the descriptor's ``tool_prefix`` stripped of the ``mcp__`` scaffolding
56+
(so tools surface consistently), falling back to ``namespace-name`` which is
57+
always present and unique per asset.
58+
"""
59+
namespace = asset.get("namespace", "")
60+
name = asset.get("name", "")
61+
return f"{namespace}-{name}".strip("-") or name or "registry-mcp"
62+
63+
64+
def apply_mcp_assets(repo_dir: str, resolved_mcp_servers: list[dict[str, Any]]) -> int:
65+
"""Merge resolved MCP server assets into ``<repo_dir>/.mcp.json``.
66+
67+
Read-merge-write on the ``mcpServers`` map, preserving any channel MCP or
68+
repo-committed entries. Each asset's descriptor carries a ``server_config``
69+
(the ``mcpServers`` entry value); assets missing it are skipped with a warn
70+
rather than writing a malformed entry.
71+
72+
Args:
73+
repo_dir: the cloned-repo working directory the SDK uses as ``cwd``.
74+
resolved_mcp_servers: the ``mcp_servers`` list from the resolved bundle;
75+
each item is a resolved-asset dict (kind/namespace/name/version/descriptor).
76+
77+
Returns:
78+
The number of MCP entries written (0 when nothing to apply, or on a
79+
write failure — logged).
80+
"""
81+
if not resolved_mcp_servers:
82+
return 0
83+
if not repo_dir or not os.path.isdir(repo_dir):
84+
log("WARN", f"apply_mcp_assets: repo_dir missing or not a directory: {repo_dir!r}")
85+
return 0
86+
87+
mcp_path = os.path.join(repo_dir, ".mcp.json")
88+
config = _read_existing_mcp_config(mcp_path)
89+
servers = config.get("mcpServers")
90+
if not isinstance(servers, dict):
91+
servers = {}
92+
93+
written = 0
94+
for asset in resolved_mcp_servers:
95+
descriptor = asset.get("descriptor") or {}
96+
server_config = descriptor.get(_MCP_SERVER_CONFIG_KEY)
97+
if not isinstance(server_config, dict):
98+
log(
99+
"WARN",
100+
f"apply_mcp_assets: asset {asset.get('namespace')}/{asset.get('name')} "
101+
f"has no '{_MCP_SERVER_CONFIG_KEY}' in its descriptor; skipping",
102+
)
103+
continue
104+
servers[_mcp_server_key(asset)] = server_config
105+
written += 1
106+
107+
if written == 0:
108+
return 0
109+
110+
config["mcpServers"] = servers
111+
try:
112+
with open(mcp_path, "w", encoding="utf-8") as f:
113+
json.dump(config, f, indent=2)
114+
f.write("\n")
115+
except OSError as e:
116+
log("ERROR", f"apply_mcp_assets: failed to write {mcp_path}: {e}")
117+
return 0
118+
119+
log("TASK", f"registry: merged {written} MCP server asset(s) into {mcp_path}")
120+
return written
121+
122+
123+
def apply_cedar_modules(resolved_cedar_modules: list[dict[str, Any]]) -> list[str]:
124+
"""Return Cedar policy text for resolved cedar_policy_module assets.
125+
126+
STUB (PR 3): the resolver already returns these assets, but wiring the text
127+
into the PolicyEngine's policy set is deferred. Returns an empty list today
128+
so callers can unconditionally extend their policy list.
129+
"""
130+
if resolved_cedar_modules:
131+
log(
132+
"TASK",
133+
f"registry: {len(resolved_cedar_modules)} cedar_policy_module asset(s) "
134+
"resolved but not yet applied (PR 3)",
135+
)
136+
return []
137+
138+
139+
def apply_skills(repo_dir: str, resolved_skills: list[dict[str, Any]]) -> int:
140+
"""Apply resolved skill assets to the SDK setting sources.
141+
142+
STUB (PR 3): returns 0 today. The resolver returns these assets; loading the
143+
prompt fragments into the SDK's setting_sources is deferred to PR 3.
144+
"""
145+
if resolved_skills:
146+
log(
147+
"TASK",
148+
f"registry: {len(resolved_skills)} skill asset(s) resolved but not yet applied (PR 3)",
149+
)
150+
return 0
151+
152+
153+
def apply_resolved_assets(repo_dir: str, resolved_assets: dict[str, list[dict]]) -> None:
154+
"""Apply an entire resolved-asset bundle to the runtime.
155+
156+
Dispatches each kind to its loader. MVP applies MCP servers; cedar/skills are
157+
logged-only stubs (PR 3). Safe to call with an empty bundle (no-op).
158+
"""
159+
if not resolved_assets:
160+
return
161+
apply_mcp_assets(repo_dir, resolved_assets.get("mcp_servers") or [])
162+
apply_cedar_modules(resolved_assets.get("cedar_policy_modules") or [])
163+
apply_skills(repo_dir, resolved_assets.get("skills") or [])

agent/src/server.py

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -391,6 +391,7 @@ def _run_task_background(
391391
branch_name: str = "",
392392
pr_number: str = "",
393393
cedar_policies: list[str] | None = None,
394+
resolved_assets: dict[str, list[dict]] | None = None,
394395
approval_timeout_s: int | None = None,
395396
initial_approvals: list[str] | None = None,
396397
initial_approval_gate_count: int = 0,
@@ -477,6 +478,7 @@ def _run_task_background(
477478
branch_name=branch_name,
478479
pr_number=pr_number,
479480
cedar_policies=cedar_policies,
481+
resolved_assets=resolved_assets,
480482
approval_timeout_s=approval_timeout_s,
481483
initial_approvals=initial_approvals,
482484
initial_approval_gate_count=initial_approval_gate_count,
@@ -531,6 +533,10 @@ def _extract_invocation_params(inp: dict, request: Request) -> dict:
531533
branch_name = inp.get("branch_name", "")
532534
pr_number = str(inp.get("pr_number", ""))
533535
cedar_policies = inp.get("cedar_policies") or []
536+
# Registry (#246): the resolved asset bundle the orchestrator stamped after
537+
# resolving the blueprint's pins. Absent when no assets were pinned; the
538+
# loader treats an empty dict as a no-op.
539+
resolved_assets = inp.get("resolved_assets") or {}
534540
# Cedar HITL (§7.3) — per-task approval defaults + seeded allowlist.
535541
# Both are forwarded verbatim to the pipeline; the engine
536542
# validates shape at construction time and raises on bad input.
@@ -637,6 +643,7 @@ def _extract_invocation_params(inp: dict, request: Request) -> dict:
637643
"branch_name": branch_name,
638644
"pr_number": pr_number,
639645
"cedar_policies": cedar_policies,
646+
"resolved_assets": resolved_assets,
640647
"approval_timeout_s": approval_timeout_s,
641648
"initial_approvals": initial_approvals,
642649
"initial_approval_gate_count": initial_approval_gate_count,

agent/tests/test_registry_grammar_corpus.py

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -29,8 +29,7 @@
2929

3030
def _load_grammar_fixtures() -> list[dict]:
3131
assert _FIXTURE_DIR.is_dir(), (
32-
f"expected fixture dir at {_FIXTURE_DIR}; "
33-
"see contracts/registry-resolution/README.md"
32+
f"expected fixture dir at {_FIXTURE_DIR}; see contracts/registry-resolution/README.md"
3433
)
3534
fixtures = []
3635
for path in sorted(_FIXTURE_DIR.glob("grammar-*.json")):
Lines changed: 104 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,104 @@
1+
"""Tests for the agent-side registry asset loader (#246)."""
2+
3+
from __future__ import annotations
4+
5+
import json
6+
import os
7+
8+
from registry_loader import (
9+
apply_cedar_modules,
10+
apply_mcp_assets,
11+
apply_resolved_assets,
12+
apply_skills,
13+
)
14+
15+
16+
def _read_mcp(repo_dir: str) -> dict:
17+
with open(os.path.join(repo_dir, ".mcp.json"), encoding="utf-8") as f:
18+
return json.load(f)
19+
20+
21+
def _mcp_asset(name: str, server_config: dict, namespace: str = "acme") -> dict:
22+
return {
23+
"kind": "mcp_server",
24+
"namespace": namespace,
25+
"name": name,
26+
"version": "1.0.0",
27+
"descriptor": {"summary": "s", "permissions": [], "server_config": server_config},
28+
"warnings": [],
29+
}
30+
31+
32+
class TestApplyMcpAssets:
33+
def test_no_op_on_empty_list(self, tmp_path):
34+
assert apply_mcp_assets(str(tmp_path), []) == 0
35+
assert not (tmp_path / ".mcp.json").exists()
36+
37+
def test_writes_resolved_server_into_mcp_json(self, tmp_path):
38+
cfg = {"type": "http", "url": "https://mcp.example/pdf"}
39+
written = apply_mcp_assets(str(tmp_path), [_mcp_asset("pdf-tools", cfg)])
40+
assert written == 1
41+
servers = _read_mcp(str(tmp_path))["mcpServers"]
42+
assert servers["acme-pdf-tools"] == cfg
43+
44+
def test_merges_alongside_existing_channel_entry(self, tmp_path):
45+
# Simulate channel_mcp.py having already written a linear-server entry.
46+
(tmp_path / ".mcp.json").write_text(
47+
json.dumps({"mcpServers": {"linear-server": {"type": "http", "url": "x"}}})
48+
)
49+
apply_mcp_assets(str(tmp_path), [_mcp_asset("pdf-tools", {"type": "http", "url": "y"})])
50+
servers = _read_mcp(str(tmp_path))["mcpServers"]
51+
# Both coexist — the channel entry is not clobbered.
52+
assert "linear-server" in servers
53+
assert "acme-pdf-tools" in servers
54+
55+
def test_writes_multiple_assets(self, tmp_path):
56+
written = apply_mcp_assets(
57+
str(tmp_path),
58+
[_mcp_asset("a", {"url": "1"}), _mcp_asset("b", {"url": "2"})],
59+
)
60+
assert written == 2
61+
assert len(_read_mcp(str(tmp_path))["mcpServers"]) == 2
62+
63+
def test_skips_asset_missing_server_config(self, tmp_path):
64+
bad = {
65+
"kind": "mcp_server",
66+
"namespace": "acme",
67+
"name": "broken",
68+
"version": "1.0.0",
69+
"descriptor": {"summary": "s", "permissions": []}, # no server_config
70+
"warnings": [],
71+
}
72+
assert apply_mcp_assets(str(tmp_path), [bad]) == 0
73+
74+
def test_no_op_on_missing_repo_dir(self):
75+
assert apply_mcp_assets("/nonexistent/dir/xyz", [_mcp_asset("a", {"url": "1"})]) == 0
76+
77+
def test_tolerates_malformed_existing_mcp_json(self, tmp_path):
78+
(tmp_path / ".mcp.json").write_text("{ not json")
79+
written = apply_mcp_assets(str(tmp_path), [_mcp_asset("pdf", {"url": "z"})])
80+
assert written == 1
81+
assert "acme-pdf" in _read_mcp(str(tmp_path))["mcpServers"]
82+
83+
84+
class TestStubs:
85+
def test_cedar_modules_stub_returns_empty(self):
86+
assert apply_cedar_modules([{"kind": "cedar_policy_module"}]) == []
87+
88+
def test_skills_stub_returns_zero(self, tmp_path):
89+
assert apply_skills(str(tmp_path), [{"kind": "skill"}]) == 0
90+
91+
92+
class TestApplyResolvedAssets:
93+
def test_empty_bundle_is_noop(self, tmp_path):
94+
apply_resolved_assets(str(tmp_path), {})
95+
assert not (tmp_path / ".mcp.json").exists()
96+
97+
def test_dispatches_mcp_servers(self, tmp_path):
98+
bundle = {
99+
"mcp_servers": [_mcp_asset("pdf-tools", {"type": "http", "url": "u"})],
100+
"cedar_policy_modules": [],
101+
"skills": [],
102+
}
103+
apply_resolved_assets(str(tmp_path), bundle)
104+
assert "acme-pdf-tools" in _read_mcp(str(tmp_path))["mcpServers"]

cdk/bootstrap/bootstrap-template.yaml

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1054,6 +1054,10 @@ Resources:
10541054
- cognito-idp:DeleteUserPoolClient
10551055
- cognito-idp:DescribeUserPoolClient
10561056
- cognito-idp:UpdateUserPoolClient
1057+
- cognito-idp:CreateGroup
1058+
- cognito-idp:DeleteGroup
1059+
- cognito-idp:GetGroup
1060+
- cognito-idp:UpdateGroup
10571061
- cognito-idp:TagResource
10581062
- cognito-idp:UntagResource
10591063
- cognito-idp:ListTagsForResource

cdk/bootstrap/policies/application.json

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -111,6 +111,10 @@
111111
"cognito-idp:DeleteUserPoolClient",
112112
"cognito-idp:DescribeUserPoolClient",
113113
"cognito-idp:UpdateUserPoolClient",
114+
"cognito-idp:CreateGroup",
115+
"cognito-idp:DeleteGroup",
116+
"cognito-idp:GetGroup",
117+
"cognito-idp:UpdateGroup",
114118
"cognito-idp:TagResource",
115119
"cognito-idp:UntagResource",
116120
"cognito-idp:ListTagsForResource",

0 commit comments

Comments
 (0)