Skip to content

Commit 8769eba

Browse files
HweinstockHweinstock
committed
fix: include upstream repo in agent prompt and SOP
1 parent 3be5a74 commit 8769eba

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

42 files changed

+9440
-1
lines changed

‎.github/agent-sops/task-tester.sop.md‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,9 @@ You have TUI harness MCP tools: `tui_launch`, `tui_send_keys`, `tui_action`, `tu
1414

1515
You also have `shell` for setup commands and GitHub tools for posting comments.
1616

17+
**Important:** Always use `aws/agentcore-cli` as the repository for all GitHub API calls (get PR, post comments, etc.),
18+
not the fork repository.
19+
1720
## Steps
1821

1922
### 1. Setup

‎.github/scripts/javascript/process-inputs.cjs‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -85,7 +85,9 @@ function buildPrompts(mode, issueId, isPullRequest, command, branchName, inputs)
8585
const systemPrompt = fs.readFileSync(scriptFile, 'utf8');
8686

8787
let prompt = isPullRequest ? 'The pull request id is:' : 'The issue id is:';
88-
prompt += `${issueId}\n${command}\nreview and continue`;
88+
prompt += `${issueId}\n`;
89+
prompt += `The repository is: aws/agentcore-cli\n`;
90+
prompt += `${command}\nreview and continue`;
8991

9092
return { sessionId, systemPrompt, prompt };
9193
}

‎E2E_FAILURE_REPORT.md‎

Lines changed: 263 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,263 @@
1+
# E2E Test Failure Report — agentcore-cli
2+
3+
**Date:** 2026-03-26
4+
**CI Run:** [#23616254746](https://github.com/aws/agentcore-cli/actions/runs/23616254746)
5+
**Branch:** main (push trigger from PR #657 merge)
6+
**Result:** 84/87 passed, 3 skipped (1 suite failed in `beforeAll`)
7+
8+
## Summary
9+
10+
The `byo-custom-jwt.test.ts` E2E test (introduced in PR #657) fails because the `e2e-github-actions` IAM role in account `685197708687` is missing **8 required IAM actions**. There are two distinct gaps:
11+
12+
1. **7 Cognito actions** — the test creates/deletes a Cognito user pool as an OIDC provider
13+
2. **1 CloudFormation action** — the test calls `GetTemplate` to verify the deployed CFN template
14+
15+
The Cognito gap causes the immediate `beforeAll` failure. The `GetTemplate` gap is a **hidden second failure** that would surface after fixing Cognito.
16+
17+
## Proven Root Causes
18+
19+
### Failure 1: Missing `cognito-idp:*` permissions (VISIBLE)
20+
21+
The test's `beforeAll` hook creates a Cognito user pool to serve as the OIDC provider for CUSTOM_JWT auth. The `e2e-github-actions` role has zero Cognito permissions.
22+
23+
**CI error:**
24+
```
25+
AccessDeniedException: User: arn:aws:sts::685197708687:assumed-role/e2e-github-actions/GitHubActions
26+
is not authorized to perform: cognito-idp:CreateUserPool on resource:
27+
arn:aws:cognito-idp:us-east-1:685197708687:userpool/*
28+
because no identity-based policy allows the cognito-idp:CreateUserPool action
29+
```
30+
31+
**Reproduction (simulate-principal-policy from Admin in 685197708687):**
32+
```
33+
+-----------------------------------+----------------+
34+
| cognito-idp:CreateUserPool | implicitDeny |
35+
| cognito-idp:CreateUserPoolDomain | implicitDeny |
36+
| cognito-idp:CreateResourceServer | implicitDeny |
37+
| cognito-idp:CreateUserPoolClient | implicitDeny |
38+
| cognito-idp:DeleteResourceServer | implicitDeny |
39+
| cognito-idp:DeleteUserPoolDomain | implicitDeny |
40+
| cognito-idp:DeleteUserPool | implicitDeny |
41+
+-----------------------------------+----------------+
42+
```
43+
44+
### Failure 2: Missing `cloudformation:GetTemplate` (HIDDEN)
45+
46+
The test's first `it` block calls `cfnClient.send(new GetTemplateCommand(...))` to verify the deployed CFN template contains `AuthorizerConfiguration`. This call uses the ambient e2e role credentials (not the CDK deploy role). The role only has `cloudformation:DescribeStacks`, not `GetTemplate`.
47+
48+
**Reproduction:**
49+
```
50+
+-----------------------------------+----------------+
51+
| cloudformation:GetTemplate | implicitDeny |
52+
+-----------------------------------+----------------+
53+
```
54+
55+
This failure is currently masked because `beforeAll` fails first (Cognito), causing all 3 tests to skip.
56+
57+
### Verification: Existing tests pass correctly
58+
59+
All actions used by the 84 passing tests are allowed:
60+
```
61+
+----------------------------------------------------+---------------+
62+
| sts:GetCallerIdentity | allowed |
63+
| cloudformation:DescribeStacks | allowed |
64+
| bedrock-agentcore:InvokeAgentRuntime | allowed |
65+
| bedrock-agentcore:GetAgentRuntime | allowed |
66+
| bedrock-agentcore:DeleteApiKeyCredentialProvider | allowed |
67+
| secretsmanager:GetSecretValue | allowed |
68+
| logs:FilterLogEvents | allowed |
69+
| tag:GetResources | allowed |
70+
+----------------------------------------------------+---------------+
71+
```
72+
73+
## Fix
74+
75+
Add the following statement to the `test-only-permissions` inline policy on the `e2e-github-actions` role in account `685197708687`:
76+
77+
```json
78+
{
79+
"Sid": "E2ECustomJwtCognitoSetup",
80+
"Effect": "Allow",
81+
"Action": [
82+
"cognito-idp:CreateUserPool",
83+
"cognito-idp:CreateUserPoolDomain",
84+
"cognito-idp:CreateResourceServer",
85+
"cognito-idp:CreateUserPoolClient",
86+
"cognito-idp:DeleteResourceServer",
87+
"cognito-idp:DeleteUserPoolDomain",
88+
"cognito-idp:DeleteUserPool"
89+
],
90+
"Resource": "*"
91+
}
92+
```
93+
94+
And add `cloudformation:GetTemplate` to the `e2e-permissions` inline policy's `CloudFormationStackStatus` statement:
95+
96+
```json
97+
{
98+
"Sid": "CloudFormationStackStatus",
99+
"Effect": "Allow",
100+
"Action": [
101+
"cloudformation:DescribeStacks",
102+
"cloudformation:GetTemplate"
103+
],
104+
"Resource": "*"
105+
}
106+
```
107+
108+
Also update `iam-policy-e2e-additions.json` in the repo to document these additions:
109+
110+
```json
111+
{
112+
"Version": "2012-10-17",
113+
"Statement": [
114+
{
115+
"Sid": "E2ECredentialCleanup",
116+
"Effect": "Allow",
117+
"Action": "bedrock-agentcore:DeleteApiKeyCredentialProvider",
118+
"Resource": "*"
119+
},
120+
{
121+
"Sid": "E2ESecretsManager",
122+
"Effect": "Allow",
123+
"Action": [
124+
"secretsmanager:GetSecretValue",
125+
"secretsmanager:CreateSecret"
126+
],
127+
"Resource": "*"
128+
},
129+
{
130+
"Sid": "E2ECustomJwtCognitoSetup",
131+
"Effect": "Allow",
132+
"Action": [
133+
"cognito-idp:CreateUserPool",
134+
"cognito-idp:CreateUserPoolDomain",
135+
"cognito-idp:CreateResourceServer",
136+
"cognito-idp:CreateUserPoolClient",
137+
"cognito-idp:DeleteResourceServer",
138+
"cognito-idp:DeleteUserPoolDomain",
139+
"cognito-idp:DeleteUserPool"
140+
],
141+
"Resource": "*"
142+
},
143+
{
144+
"Sid": "E2ECloudFormationTemplateVerification",
145+
"Effect": "Allow",
146+
"Action": "cloudformation:GetTemplate",
147+
"Resource": "*"
148+
}
149+
]
150+
}
151+
```
152+
153+
## How to Apply the Fix (685197708687)
154+
155+
```bash
156+
# Assume Admin in the E2E account
157+
ada credentials update --account 685197708687 --provider isengard --role Admin --once
158+
159+
# 1. Add Cognito permissions to test-only-permissions
160+
aws iam put-role-policy \
161+
--role-name e2e-github-actions \
162+
--policy-name test-only-permissions \
163+
--policy-document '{
164+
"Statement": [
165+
{
166+
"Sid": "Statement1",
167+
"Effect": "Allow",
168+
"Action": ["bedrock-agentcore:DeleteApiKeyCredentialProvider"],
169+
"Resource": "*"
170+
},
171+
{
172+
"Sid": "Statement2",
173+
"Effect": "Allow",
174+
"Action": ["secretsmanager:GetSecretValue", "secretsmanager:CreateSecret"],
175+
"Resource": ["*"]
176+
},
177+
{
178+
"Sid": "E2ECustomJwtCognitoSetup",
179+
"Effect": "Allow",
180+
"Action": [
181+
"cognito-idp:CreateUserPool",
182+
"cognito-idp:CreateUserPoolDomain",
183+
"cognito-idp:CreateResourceServer",
184+
"cognito-idp:CreateUserPoolClient",
185+
"cognito-idp:DeleteResourceServer",
186+
"cognito-idp:DeleteUserPoolDomain",
187+
"cognito-idp:DeleteUserPool"
188+
],
189+
"Resource": "*"
190+
}
191+
]
192+
}'
193+
194+
# 2. Add GetTemplate to e2e-permissions (full policy — must include all existing statements)
195+
# Get current policy first, add cloudformation:GetTemplate to CloudFormationStackStatus, then put back
196+
aws iam get-role-policy --role-name e2e-github-actions --policy-name e2e-permissions \
197+
--query PolicyDocument > /tmp/e2e-permissions.json
198+
199+
# Edit /tmp/e2e-permissions.json: add "cloudformation:GetTemplate" to the CloudFormationStackStatus statement
200+
# Then apply:
201+
aws iam put-role-policy \
202+
--role-name e2e-github-actions \
203+
--policy-name e2e-permissions \
204+
--policy-document file:///tmp/e2e-permissions.json
205+
206+
# 3. Verify the fix
207+
aws iam simulate-principal-policy \
208+
--policy-source-arn arn:aws:iam::685197708687:role/e2e-github-actions \
209+
--action-names \
210+
cognito-idp:CreateUserPool \
211+
cognito-idp:CreateUserPoolDomain \
212+
cognito-idp:CreateResourceServer \
213+
cognito-idp:CreateUserPoolClient \
214+
cognito-idp:DeleteResourceServer \
215+
cognito-idp:DeleteUserPoolDomain \
216+
cognito-idp:DeleteUserPool \
217+
cloudformation:GetTemplate \
218+
--output table \
219+
--query 'EvaluationResults[*].[EvalActionName,EvalDecision]'
220+
# Expected: all "allowed"
221+
```
222+
223+
## Reproducing in Dev Account (440744214761)
224+
225+
The `byo-custom-jwt.test.ts` test can run in any account with sufficient permissions. Prerequisites:
226+
227+
```bash
228+
# 1. Assume credentials with Admin/PowerUser in your dev account
229+
ada credentials update --account 440744214761 --provider isengard --role hkobew-default --once
230+
231+
# 2. Build the CLI
232+
cd ~/repos/agentcore-cli
233+
npm ci && npm run build
234+
235+
# 3. Build the CDK tarball (required — test checks CDK_TARBALL env var)
236+
# Clone the CDK repo, build, and pack:
237+
CDK_REPO="<CDK_REPO_NAME from GitHub secrets>"
238+
git clone https://github.com/${CDK_REPO}.git /tmp/agentcore-cdk
239+
cd /tmp/agentcore-cdk && npm ci && npm run build && npm pack
240+
export CDK_TARBALL=$(ls /tmp/agentcore-cdk/*.tgz | head -1)
241+
242+
# 4. Set required env vars
243+
export AWS_REGION=us-east-1
244+
export AWS_ACCOUNT_ID=440744214761
245+
246+
# 5. Run just the failing test
247+
cd ~/repos/agentcore-cli
248+
npx vitest run --project e2e e2e-tests/byo-custom-jwt.test.ts
249+
```
250+
251+
Your dev account's `hkobew-default` role likely has sufficient Cognito and CloudFormation permissions already (unlike the locked-down e2e-github-actions role).
252+
253+
## Other CI Failures (Not on Main)
254+
255+
These are on PR branches, not main. Included for completeness:
256+
257+
| Run | Branch | Cause | Fix |
258+
|-----|--------|-------|-----|
259+
| 23618035513 | episodic-memory-strategy | Prettier violation in `docs/memory.md` | Run `npx prettier --write docs/memory.md` |
260+
| 23616427107 | dependabot/eslint-10 | `eslint-plugin-import@2.32.0` incompatible with eslint 10 | Close the dependabot PR; wait for plugin update |
261+
| 23616334309 | json-schema-gen | Prettier violation in `schemas/README.md` | Run `npx prettier --write schemas/README.md` |
262+
263+
None of these affect main branch CI.

0 commit comments

Comments
 (0)