Skip to content

Commit a4187f8

Browse files
aidandaly24aidandaly24
committed
docs: add documentation for features merged after initial audit
Cover 13 PRs merged to main since the initial docs audit (March 28): - import subcommands: runtime, memory, evaluator, online-eval (#763, #780) - invoke/dev exec mode with --exec and --timeout (#750) - code-based evaluator support with --type, --lambda-arn, --timeout (#739) - ground truth eval inputs: --assertion, --expected-trajectory, --expected-response (#732) - memory record streaming: --data-stream-arn, --stream-content-level (#531, #534) - create --skip-install flag (#782) - fetch access --identity-name option (#774)
1 parent d6a02bf commit a4187f8

File tree

4 files changed

+293
-39
lines changed

4 files changed

+293
-39
lines changed

‎docs/commands.md‎

Lines changed: 137 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -89,6 +89,7 @@ agentcore create \
8989
| `--output-dir <dir>` | Output directory |
9090
| `--skip-git` | Skip git initialization |
9191
| `--skip-python-setup` | Skip venv setup |
92+
| `--skip-install` | Skip all dependency installation (npm install, uv sync) |
9293
| `--dry-run` | Preview without creating |
9394
| `--json` | JSON output |
9495

@@ -150,6 +151,88 @@ agentcore validate -d ./my-project
150151
| ------------------------ | ----------------- |
151152
| `-d, --directory <path>` | Project directory |
152153

154+
### import
155+
156+
Import resources from AWS or from a starter toolkit YAML.
157+
158+
```bash
159+
# Interactive TUI wizard
160+
agentcore import
161+
162+
# Import from starter toolkit YAML
163+
agentcore import --source .bedrock_agentcore.yaml
164+
agentcore import --source .bedrock_agentcore.yaml --target staging -y
165+
```
166+
167+
| Flag | Description |
168+
| ----------------- | ------------------------------------------------------------ |
169+
| `--source <path>` | Path to `.bedrock_agentcore.yaml` configuration file |
170+
| `--target <name>` | Deployment target (only needed if project has multiple) |
171+
| `-y, --yes` | Auto-confirm prompts |
172+
173+
#### import runtime
174+
175+
Import an existing AgentCore Runtime from your AWS account.
176+
177+
```bash
178+
agentcore import runtime
179+
agentcore import runtime --arn arn:aws:bedrock-agentcore:us-east-1:123456789012:runtime/abc123 --code ./src
180+
agentcore import runtime --arn ... --code ./src --entrypoint handler.py --name MyAgent -y
181+
```
182+
183+
| Flag | Description |
184+
| --------------------- | ------------------------------------------------------------------- |
185+
| `--arn <runtimeArn>` | Runtime ARN to import |
186+
| `--code <path>` | Path to directory containing the entrypoint file |
187+
| `--entrypoint <file>` | Entrypoint file (auto-detected from runtime if omitted, e.g. `main.py`) |
188+
| `--name <name>` | Local name for the imported runtime |
189+
| `-y, --yes` | Auto-confirm prompts |
190+
191+
#### import memory
192+
193+
Import an existing AgentCore Memory from your AWS account.
194+
195+
```bash
196+
agentcore import memory
197+
agentcore import memory --arn arn:aws:bedrock-agentcore:us-east-1:123456789012:memory/def456 --name SharedMemory -y
198+
```
199+
200+
| Flag | Description |
201+
| ------------------- | ---------------------------------- |
202+
| `--arn <memoryArn>` | Memory ARN to import |
203+
| `--name <name>` | Local name for the imported memory |
204+
| `-y, --yes` | Auto-confirm prompts |
205+
206+
#### import evaluator
207+
208+
Import an existing AgentCore Evaluator from your AWS account.
209+
210+
```bash
211+
agentcore import evaluator
212+
agentcore import evaluator --arn arn:aws:bedrock-agentcore:us-east-1:123456789012:evaluator/ghi789 --name MyEvaluator -y
213+
```
214+
215+
| Flag | Description |
216+
| ---------------------- | ------------------------------------- |
217+
| `--arn <evaluatorArn>` | Evaluator ARN to import |
218+
| `--name <name>` | Local name for the imported evaluator |
219+
| `-y, --yes` | Auto-confirm prompts |
220+
221+
#### import online-eval
222+
223+
Import an existing AgentCore Online Evaluation Config from your AWS account.
224+
225+
```bash
226+
agentcore import online-eval
227+
agentcore import online-eval --arn arn:aws:bedrock-agentcore:us-east-1:123456789012:online-eval-config/jkl012 --name QualityMonitor -y
228+
```
229+
230+
| Flag | Description |
231+
| ----------------- | ---------------------------------------------- |
232+
| `--arn <configArn>` | Online evaluation config ARN to import |
233+
| `--name <name>` | Local name for the imported online eval config |
234+
| `-y, --yes` | Auto-confirm prompts |
235+
153236
---
154237

155238
## Resource Management
@@ -239,10 +322,14 @@ agentcore add memory \
239322

240323
| Flag | Description |
241324
| ---------------------- | --------------------------------------------------------------------------- |
242-
| `--name <name>` | Memory name |
243-
| `--strategies <types>` | Comma-separated: `SEMANTIC`, `SUMMARIZATION`, `USER_PREFERENCE`, `EPISODIC` |
244-
| `--expiry <days>` | Event expiry duration in days (default: 30, min: 7, max: 365) |
245-
| `--json` | JSON output |
325+
| `--name <name>` | Memory name |
326+
| `--strategies <types>` | Comma-separated: `SEMANTIC`, `SUMMARIZATION`, `USER_PREFERENCE`, `EPISODIC` |
327+
| `--expiry <days>` | Event expiry duration in days (default: 30, min: 7, max: 365) |
328+
| `--data-stream-arn <arn>` | Kinesis data stream ARN for memory record streaming |
329+
| `--stream-content-level <level>` | Stream content level: `FULL_CONTENT` or `METADATA_ONLY` |
330+
| `--delivery-type <type>` | Delivery target type (default: `kinesis`) |
331+
| `--stream-delivery-resources <json>` | Stream delivery config as JSON (advanced, overrides flat flags) |
332+
| `--json` | JSON output |
246333

247334
### add gateway
248335

@@ -409,24 +496,42 @@ agentcore add credential \
409496

410497
### add evaluator
411498

412-
Add a custom LLM-as-a-Judge evaluator. See [Evaluations](evals.md) for full details.
499+
Add an evaluator. Supports LLM-as-a-Judge (default) and code-based evaluator types. See [Evaluations](evals.md) for full details.
413500

414501
```bash
502+
# LLM-as-a-Judge evaluator
415503
agentcore add evaluator \
416504
--name ResponseQuality \
417505
--level SESSION \
418506
--model us.anthropic.claude-sonnet-4-5-20250514-v1:0 \
419507
--instructions "Evaluate the response quality. Context: {context}" \
420508
--rating-scale 1-5-quality
509+
510+
# Code-based evaluator (managed — scaffolds a Python Lambda)
511+
agentcore add evaluator \
512+
--name CustomScorer \
513+
--type code-based \
514+
--level TRACE
515+
516+
# Code-based evaluator (external — uses existing Lambda)
517+
agentcore add evaluator \
518+
--name ExternalScorer \
519+
--type code-based \
520+
--level TRACE \
521+
--lambda-arn arn:aws:lambda:us-east-1:123456789012:function:my-scorer \
522+
--timeout 120
421523
```
422524

423525
| Flag | Description |
424526
| ------------------------- | -------------------------------------------------------------------------- |
425527
| `--name <name>` | Evaluator name |
426528
| `--level <level>` | `SESSION`, `TRACE`, or `TOOL_CALL` |
427-
| `--model <model>` | Bedrock model ID for the LLM judge |
428-
| `--instructions <text>` | Evaluation prompt with placeholders (e.g. `{context}`) |
429-
| `--rating-scale <preset>` | `1-5-quality`, `1-3-simple`, `pass-fail`, `good-neutral-bad`, or custom |
529+
| `--type <type>` | `llm-as-a-judge` (default) or `code-based` |
530+
| `--model <model>` | [LLM] Bedrock model ID for the LLM judge |
531+
| `--instructions <text>` | [LLM] Evaluation prompt with placeholders (e.g. `{context}`) |
532+
| `--rating-scale <preset>` | [LLM] `1-5-quality`, `1-3-simple`, `pass-fail`, `good-neutral-bad`, or custom |
533+
| `--lambda-arn <arn>` | [Code-based] Existing Lambda function ARN (external) |
534+
| `--timeout <seconds>` | [Code-based] Lambda timeout in seconds, 1–300 (default: 60) |
430535
| `--config <path>` | Config JSON file (overrides `--model`, `--instructions`, `--rating-scale`) |
431536
| `--json` | JSON output |
432537

@@ -491,6 +596,7 @@ agentcore dev --runtime MyAgent --port 3000
491596
agentcore dev --logs # Non-interactive
492597
agentcore dev "Hello" --stream # Invoke running dev server
493598
agentcore dev "Hello" --runtime MyAgent # Invoke specific runtime
599+
agentcore dev --exec "pip list" # Run command in dev container
494600

495601
# MCP protocol dev commands
496602
agentcore dev list-tools
@@ -504,6 +610,7 @@ agentcore dev call-tool --tool myTool --input '{"arg": "value"}'
504610
| `-r, --runtime <name>` | Runtime to run or invoke (required if multiple runtimes) |
505611
| `-s, --stream` | Stream response when invoking |
506612
| `-l, --logs` | Non-interactive stdout logging |
613+
| `--exec` | Execute a shell command in the running dev container (Container agents only) |
507614
| `--tool <name>` | MCP tool name (with `call-tool` prompt) |
508615
| `--input <json>` | MCP tool arguments as JSON (with `--tool`) |
509616
| `-H, --header <h>` | Custom header (`"Name: Value"`, repeatable) |
@@ -518,6 +625,8 @@ agentcore invoke --prompt "Hello" --stream
518625
agentcore invoke --runtime MyAgent --target staging
519626
agentcore invoke --session-id abc123 # Continue session
520627
agentcore invoke --json # JSON output
628+
agentcore invoke --exec "pip list" # Run shell command in runtime container
629+
agentcore invoke --exec "python script.py" --timeout 30
521630

522631
# MCP protocol invoke
523632
agentcore invoke call-tool --tool myTool --input '{"key": "value"}'
@@ -532,6 +641,8 @@ agentcore invoke call-tool --tool myTool --input '{"key": "value"}'
532641
| `--session-id <id>` | Continue a specific session |
533642
| `--user-id <id>` | User ID for runtime invocation (default: `default-user`) |
534643
| `--stream` | Stream response in real-time |
644+
| `--exec` | Execute a shell command in the runtime container |
645+
| `--timeout <seconds>`| Timeout in seconds for `--exec` commands |
535646
| `--tool <name>` | MCP tool name (use with `call-tool` prompt) |
536647
| `--input <json>` | MCP tool arguments as JSON (use with `--tool`) |
537648
| `-H, --header <h>` | Custom header (`"Name: Value"`, repeatable) |
@@ -621,16 +732,19 @@ agentcore run eval \
621732

622733
| Flag | Description |
623734
| ---------------------------- | ------------------------------------------- |
624-
| `-r, --runtime <name>` | Runtime name from project |
625-
| `--runtime-arn <arn>` | Runtime ARN (standalone mode) |
626-
| `-e, --evaluator <names...>` | Evaluator name(s) or `Builtin.*` IDs |
627-
| `--evaluator-arn <arns...>` | Evaluator ARN(s) (use with `--runtime-arn`) |
628-
| `--region <region>` | AWS region (required with `--runtime-arn`) |
629-
| `-s, --session-id <id>` | Evaluate a specific session |
630-
| `-t, --trace-id <id>` | Evaluate a specific trace |
631-
| `--days <days>` | Lookback window in days (default: 7) |
632-
| `--output <path>` | Custom output file path |
633-
| `--json` | JSON output |
735+
| `-r, --runtime <name>` | Runtime name from project |
736+
| `--runtime-arn <arn>` | Runtime ARN (standalone mode) |
737+
| `-e, --evaluator <names...>` | Evaluator name(s) or `Builtin.*` IDs |
738+
| `--evaluator-arn <arns...>` | Evaluator ARN(s) (use with `--runtime-arn`) |
739+
| `--region <region>` | AWS region (required with `--runtime-arn`) |
740+
| `-s, --session-id <id>` | Evaluate a specific session |
741+
| `-t, --trace-id <id>` | Evaluate a specific trace |
742+
| `--days <days>` | Lookback window in days (default: 7) |
743+
| `-A, --assertion <text...>` | Ground truth assertion (repeatable) |
744+
| `--expected-trajectory <names>` | Expected tool calls in order (comma-separated) |
745+
| `--expected-response <text>` | Expected agent response text |
746+
| `--output <path>` | Custom output file path |
747+
| `--json` | JSON output |
634748

635749
### evals history
636750

@@ -713,10 +827,11 @@ agentcore fetch access --name MyAgent --type agent --target staging
713827

714828
| Flag | Description |
715829
| ----------------- | --------------------------------------------- |
716-
| `--name <name>` | Gateway or agent name |
717-
| `--type <type>` | Resource type: `gateway` (default) or `agent` |
718-
| `--target <name>` | Deployment target |
719-
| `--json` | JSON output |
830+
| `--name <name>` | Gateway or agent name |
831+
| `--type <type>` | Resource type: `gateway` (default) or `agent` |
832+
| `--target <name>` | Deployment target |
833+
| `--identity-name <name>` | Identity credential name for token fetch |
834+
| `--json` | JSON output |
720835

721836
### package
722837

0 commit comments

Comments
 (0)