Skip to content

Commit cf30991

Browse files
committed
chore: sync main into preview — evo features + harness support
Merge public/main into preview. Resolves conflicts by keeping both: - Preview: harness support - Main: evo features (config bundles, batch eval, recommendations, AB tests)
2 parents 1277430 + 90924d6 commit cf30991

262 files changed

Lines changed: 30060 additions & 890 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.github/harness/prompts/review.md

Lines changed: 8 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -3,11 +3,16 @@ Review this GitHub PR: {pr_url}
33
You have tools to fetch the PR diff, read files, search the web, and post comments on the PR.
44

55
You have these repos cloned locally for context:
6+
67
- /opt/workspace/agentcore-cli — aws/agentcore-cli
78
- /opt/workspace/agentcore-l3-cdk-constructs — aws/agentcore-l3-cdk-constructs
89

9-
Before reviewing, read all existing comments on the PR to understand what has already been discussed. Do not repeat or re-post issues that have already been raised in existing comments.
10+
Before reviewing, read all existing comments on the PR to understand what has already been discussed. Do not repeat or
11+
re-post issues that have already been raised in existing comments.
1012

11-
Review the PR. If there are any serious issues that require code changes before merging, post a comment on the PR for each issue explaining the problem. If there are multiple ways to fix an issue, list the options so the author can choose. Skip style nits and minor suggestions — only flag things that actually need to change.
13+
Review the PR. If there are any serious issues that require code changes before merging, post a comment on the PR for
14+
each issue explaining the problem. If there are multiple ways to fix an issue, list the options so the author can
15+
choose. Skip style nits and minor suggestions — only flag things that actually need to change.
1216

13-
If all serious issues have already been raised in existing comments, or if you found no new issues, post a single comment on the PR saying it looks good to merge (or that all issues have already been flagged).
17+
If all serious issues have already been raised in existing comments, or if you found no new issues, post a single
18+
comment on the PR saying it looks good to merge (or that all issues have already been flagged).

.github/harness/prompts/system.md

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -6,16 +6,20 @@ This workspace contains two repos for developing and testing the AgentCore CLI.
66

77
### agentcore-cli/ (`aws/agentcore-cli`)
88

9-
The terminal experience for creating, developing, and deploying AI agents to AgentCore. Node.js/TypeScript CLI built with Ink (React-based TUI).
9+
The terminal experience for creating, developing, and deploying AI agents to AgentCore. Node.js/TypeScript CLI built
10+
with Ink (React-based TUI).
1011

1112
### agentcore-l3-cdk-constructs/ (`aws/agentcore-l3-cdk-constructs`)
1213

13-
AWS CDK L3 constructs for declaring and deploying AgentCore infrastructure. Used by agentcore-cli to vend CDK projects when users run `agentcore create`.
14+
AWS CDK L3 constructs for declaring and deploying AgentCore infrastructure. Used by agentcore-cli to vend CDK projects
15+
when users run `agentcore create`.
1416

1517
## How they relate
1618

1719
`agentcore-cli` is the main product. It vends CDK projects using constructs from `agentcore-l3-cdk-constructs`.
1820

1921
## Testing with a bundled distribution
2022

21-
Run `npm run bundle` in `agentcore-cli/` to create a tar distribution that includes the packaged `agentcore-l3-cdk-constructs`. You can then install it globally with `npm install -g <path-to-tar>` to test the CLI end-to-end.
23+
Run `npm run bundle` in `agentcore-cli/` to create a tar distribution that includes the packaged
24+
`agentcore-l3-cdk-constructs`. You can then install it globally with `npm install -g <path-to-tar>` to test the CLI
25+
end-to-end.

.github/scripts/prompts/review.md

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
Review this GitHub PR: {pr_url}
2+
3+
You have tools to fetch the PR diff, read files, search the web, and post comments on the PR.
4+
5+
You have these repos cloned locally for context:
6+
- /opt/workspace/agentcore-cli — aws/agentcore-cli
7+
- /opt/workspace/agentcore-l3-cdk-constructs — aws/agentcore-l3-cdk-constructs
8+
9+
Before reviewing, read all existing comments on the PR to understand what has already been discussed. Do not repeat or re-post issues that have already been raised in existing comments.
10+
11+
Review the PR. If there are any serious issues that require code changes before merging, post a comment on the PR for each issue explaining the problem. If there are multiple ways to fix an issue, list the options so the author can choose. Skip style nits and minor suggestions — only flag things that actually need to change.
12+
13+
If all serious issues have already been raised in existing comments, or if you found no new issues, post a single comment on the PR saying it looks good to merge (or that all issues have already been flagged).

.github/scripts/prompts/system.md

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
# AgentCore CLI Development Workspace
2+
3+
This workspace contains two repos for developing and testing the AgentCore CLI.
4+
5+
## Repositories
6+
7+
### agentcore-cli/ (`aws/agentcore-cli`)
8+
9+
The terminal experience for creating, developing, and deploying AI agents to AgentCore. Node.js/TypeScript CLI built with Ink (React-based TUI).
10+
11+
### agentcore-l3-cdk-constructs/ (`aws/agentcore-l3-cdk-constructs`)
12+
13+
AWS CDK L3 constructs for declaring and deploying AgentCore infrastructure. Used by agentcore-cli to vend CDK projects when users run `agentcore create`.
14+
15+
## How they relate
16+
17+
`agentcore-cli` is the main product. It vends CDK projects using constructs from `agentcore-l3-cdk-constructs`.
18+
19+
## Testing with a bundled distribution
20+
21+
Run `npm run bundle` in `agentcore-cli/` to create a tar distribution that includes the packaged `agentcore-l3-cdk-constructs`. You can then install it globally with `npm install -g <path-to-tar>` to test the CLI end-to-end.
Lines changed: 217 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,217 @@
1+
"""Invoke Bedrock AgentCore Harness to review a GitHub PR.
2+
3+
Reads PR_URL from the environment. Streams harness output to stdout.
4+
Uses raw HTTP with SigV4 signing — no custom service model needed.
5+
"""
6+
7+
import json
8+
import os
9+
import sys
10+
import time
11+
import uuid
12+
13+
import boto3
14+
from botocore.auth import SigV4Auth
15+
from botocore.awsrequest import AWSRequest
16+
from botocore.eventstream import EventStreamBuffer
17+
from urllib.parse import quote
18+
import urllib3
19+
20+
# ANSI color codes
21+
CYAN = "\033[36m"
22+
YELLOW = "\033[33m"
23+
GREEN = "\033[32m"
24+
RED = "\033[31m"
25+
DIM = "\033[2m"
26+
RESET = "\033[0m"
27+
28+
SCRIPTS_DIR = os.path.join(os.path.dirname(__file__), "..")
29+
30+
31+
def read_prompt(filename):
32+
"""Read a prompt template from the prompts directory."""
33+
path = os.path.join(SCRIPTS_DIR, "prompts", filename)
34+
with open(path) as f:
35+
return f.read()
36+
37+
38+
def invoke_harness(harness_arn, body, region):
39+
"""Send a SigV4-signed request to the harness invoke endpoint. Returns a streaming response.
40+
41+
InvokeHarness is not in standard boto3, so we call the REST API directly.
42+
boto3 is only used to resolve AWS credentials (from env vars, OIDC, etc.)
43+
and sign the request with SigV4. The response is an AWS binary event stream.
44+
"""
45+
session = boto3.Session(region_name=region)
46+
credentials = session.get_credentials().get_frozen_credentials()
47+
url = f"https://bedrock-agentcore.{region}.amazonaws.com/harnesses/invoke?harnessArn={quote(harness_arn, safe='')}"
48+
request = AWSRequest(method="POST", url=url, data=body, headers={
49+
"Content-Type": "application/json",
50+
"Accept": "application/vnd.amazon.eventstream",
51+
})
52+
SigV4Auth(credentials, "bedrock-agentcore", region).add_auth(request)
53+
return urllib3.PoolManager().urlopen(
54+
"POST", url, body=body,
55+
headers=dict(request.headers),
56+
preload_content=False,
57+
timeout=urllib3.Timeout(connect=10, read=600),
58+
)
59+
60+
61+
def parse_events(http_response):
62+
"""Yield (event_type, payload) tuples from the harness binary event stream.
63+
64+
The response arrives as raw bytes in AWS binary event stream format.
65+
EventStreamBuffer reassembles complete events from the 4KB chunks,
66+
and we decode each event's JSON payload before yielding it.
67+
"""
68+
event_buffer = EventStreamBuffer()
69+
for chunk in http_response.stream(4096):
70+
event_buffer.add_data(chunk)
71+
for event in event_buffer:
72+
if event.headers.get(":message-type") == "exception":
73+
payload = json.loads(event.payload.decode("utf-8"))
74+
print(f"\n{RED}ERROR: {payload}{RESET}", file=sys.stderr)
75+
sys.exit(1)
76+
event_type = event.headers.get(":event-type", "")
77+
if event.payload:
78+
yield event_type, json.loads(event.payload.decode("utf-8"))
79+
80+
81+
def print_stream(http_response):
82+
"""Display harness events with GitHub Actions log groups.
83+
84+
The harness streams events as the agent works:
85+
contentBlockStart — a new block begins (text or tool call)
86+
contentBlockDelta — incremental chunks of text or tool input JSON
87+
contentBlockStop — block complete, we now have full tool input to display
88+
messageStop — agent finished
89+
internalServerException — server error
90+
91+
Tool calls are wrapped in ::group::/::endgroup:: for collapsible sections
92+
in the GitHub Actions log UI. Agent reasoning text is printed inline in dim.
93+
"""
94+
start_time = time.time()
95+
iteration = 0
96+
tool_name = None
97+
tool_input = ""
98+
tool_start = 0.0
99+
in_group = False
100+
text_buffer = ""
101+
102+
def close_group():
103+
nonlocal in_group
104+
if in_group:
105+
print("::endgroup::", flush=True)
106+
in_group = False
107+
108+
def flush_text():
109+
nonlocal text_buffer
110+
if text_buffer:
111+
for line in text_buffer.splitlines():
112+
print(f"{DIM}{line}{RESET}", flush=True)
113+
text_buffer = ""
114+
115+
for event_type, payload in parse_events(http_response):
116+
117+
if event_type == "contentBlockStart":
118+
start = payload.get("start", {})
119+
if "toolUse" in start:
120+
tool_name = start["toolUse"].get("name", "unknown")
121+
tool_input = ""
122+
tool_start = time.time()
123+
iteration += 1
124+
125+
elif event_type == "contentBlockDelta":
126+
delta = payload.get("delta", {})
127+
if "text" in delta:
128+
close_group()
129+
text_buffer += delta["text"]
130+
if "toolUse" in delta:
131+
tool_input += delta["toolUse"].get("input", "")
132+
133+
elif event_type == "contentBlockStop":
134+
flush_text()
135+
if tool_name:
136+
elapsed = time.time() - tool_start
137+
try:
138+
parsed = json.loads(tool_input)
139+
except (json.JSONDecodeError, TypeError):
140+
parsed = tool_input
141+
142+
close_group()
143+
144+
cmd = parsed.get("command") if isinstance(parsed, dict) else None
145+
header = f"{CYAN}[{iteration}]{RESET} {YELLOW}{tool_name}{RESET} {DIM}({elapsed:.1f}s){RESET}"
146+
if cmd:
147+
header += f": $ {cmd}"
148+
149+
print(f"::group::{header}", flush=True)
150+
in_group = True
151+
152+
if isinstance(parsed, dict):
153+
for k, v in parsed.items():
154+
if k != "command":
155+
print(f" {DIM}{k}:{RESET} {str(v)[:300]}", flush=True)
156+
157+
tool_name = None
158+
tool_input = ""
159+
160+
elif event_type == "messageStop":
161+
flush_text()
162+
close_group()
163+
if payload.get("stopReason") == "end_turn":
164+
total = time.time() - start_time
165+
print(f"\n\n{GREEN}{'=' * 50}", flush=True)
166+
print(f" Done ({int(total // 60)}m {int(total % 60)}s)", flush=True)
167+
print(f"{'=' * 50}{RESET}", flush=True)
168+
169+
elif event_type == "internalServerException":
170+
close_group()
171+
print(f"\n{RED}ERROR: {payload}{RESET}", file=sys.stderr)
172+
sys.exit(1)
173+
174+
close_group()
175+
total = time.time() - start_time
176+
print(f"\n{GREEN}Review complete.{RESET} {DIM}({iteration} tool calls, {int(total)}s total){RESET}")
177+
178+
179+
# --- Main ---
180+
181+
# All config comes from environment variables (set via GitHub secrets/workflow)
182+
MODEL_ID = os.environ.get("HARNESS_MODEL_ID", "us.anthropic.claude-opus-4-7")
183+
HARNESS_ARN = os.environ.get("HARNESS_ARN", "")
184+
PR_URL = os.environ.get("PR_URL", "")
185+
186+
for name, val in [("HARNESS_ARN", HARNESS_ARN), ("PR_URL", PR_URL)]:
187+
if not val:
188+
print(f"{RED}ERROR: {name} environment variable is required{RESET}", file=sys.stderr)
189+
sys.exit(1)
190+
191+
# Extract region from the ARN (arn:aws:bedrock-agentcore:{region}:{account}:harness/{id})
192+
REGION = HARNESS_ARN.split(":")[3]
193+
SESSION_ID = str(uuid.uuid4()).upper()
194+
195+
print(f"{CYAN}Session:{RESET} {SESSION_ID}")
196+
print(f"{CYAN}PR:{RESET} {PR_URL}")
197+
print(f"{CYAN}Harness:{RESET} {HARNESS_ARN}")
198+
print()
199+
200+
SYSTEM_PROMPT = read_prompt("system.md")
201+
REVIEW_PROMPT = read_prompt("review.md").format(pr_url=PR_URL)
202+
203+
request_body = json.dumps({
204+
"runtimeSessionId": SESSION_ID,
205+
"systemPrompt": [{"text": SYSTEM_PROMPT}],
206+
"messages": [{"role": "user", "content": [{"text": REVIEW_PROMPT}]}],
207+
"model": {"bedrockModelConfig": {"modelId": MODEL_ID}},
208+
})
209+
210+
http_response = invoke_harness(HARNESS_ARN, request_body, REGION)
211+
212+
if http_response.status != 200:
213+
error = http_response.read().decode("utf-8")
214+
print(f"{RED}ERROR: HTTP {http_response.status}: {error}{RESET}", file=sys.stderr)
215+
sys.exit(1)
216+
217+
print_stream(http_response)

.github/workflows/e2e-tests-full.yml

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,7 @@ jobs:
2727
fail-fast: false
2828
matrix:
2929
cdk-source: [npm, main]
30+
shard: ['1/6', '2/6', '3/6', '4/6', '5/6', '6/6']
3031
steps:
3132
- uses: actions/checkout@v6
3233
with:
@@ -70,15 +71,15 @@ jobs:
7071
CDK_REPO: ${{ secrets.CDK_REPO_NAME }}
7172
- name: Install CLI globally
7273
run: npm install -g "$(npm pack | tail -1)"
73-
- name: Run E2E tests (${{ matrix.cdk-source }})
74+
- name: Run E2E tests (${{ matrix.cdk-source }}, shard ${{ matrix.shard }})
7475
env:
7576
AWS_ACCOUNT_ID: ${{ steps.aws.outputs.account_id }}
7677
AWS_REGION: ${{ inputs.aws_region || 'us-east-1' }}
7778
ANTHROPIC_API_KEY: ${{ env.E2E_ANTHROPIC_API_KEY }}
7879
OPENAI_API_KEY: ${{ env.E2E_OPENAI_API_KEY }}
7980
GEMINI_API_KEY: ${{ env.E2E_GEMINI_API_KEY }}
8081
CDK_TARBALL: ${{ env.CDK_TARBALL }}
81-
run: npm run test:e2e
82+
run: npx vitest run --project e2e --shard=${{ matrix.shard }}
8283
browser-tests:
8384
runs-on: ubuntu-latest
8485
environment: e2e-testing

.github/workflows/slack-issue-notification.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ jobs:
99
runs-on: ubuntu-latest
1010
steps:
1111
- name: Send issue details to Slack
12-
uses: slackapi/slack-github-action@v3.0.2
12+
uses: slackapi/slack-github-action@v2.1.1
1313
with:
1414
webhook: ${{ secrets.SLACK_WEBHOOK_URL }}
1515
webhook-type: webhook-trigger

.github/workflows/slack-open-prs-notification.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -40,7 +40,7 @@ jobs:
4040
);
4141
4242
- name: Send open PRs summary to Slack
43-
uses: slackapi/slack-github-action@v3.0.2
43+
uses: slackapi/slack-github-action@v2.1.1
4444
with:
4545
webhook: ${{ secrets.SLACK_OPEN_PRS_WEBHOOK_URL }}
4646
webhook-type: webhook-trigger

0 commit comments

Comments
 (0)