Skip to content

Commit 4f4dddf

Browse files
committed
Merge branch 'develop' of ssh.gitlab.aws.dev:genaiic-reusable-assets/engagement-artifacts/genaiic-idp-accelerator into develop
2 parents 5724612 + 8ce212c commit 4f4dddf

28 files changed

Lines changed: 4119 additions & 15 deletions

.gitlab-ci.yml

Lines changed: 3 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -42,10 +42,9 @@ developer_tests:
4242
# Create virtual environment
4343
- uv venv .venv
4444
- source .venv/bin/activate
45-
# Install Node.js and npm for basedpyright
46-
- curl -fsSL https://deb.nodesource.com/setup_22.x | bash -
47-
- apt-get install -y nodejs
48-
- npm install -g npm@11
45+
# Install pinned Node.js 22.14.0 and npm 11.1.0 for basedpyright
46+
- curl -fsSL https://nodejs.org/dist/v22.14.0/node-v22.14.0-linux-x64.tar.xz | tar -xJ -C /usr/local --strip-components=1
47+
- npm install -g npm@11.1.0 || { echo "npm upgrade failed"; exit 1; }
4948
- npm install -g basedpyright
5049
- uv pip install ruff
5150
# Install dependencies needed by publish.py for test imports

CHANGELOG.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,8 @@ SPDX-License-Identifier: MIT-0
77

88
### Added
99

10+
- **IDP CLI `chat` Command & SDK `ChatOperation`** — Interactive Agent Companion Chat from the terminal and programmatic SDK access. Runs the same multi-agent orchestrator as the Web UI locally, with real-time streaming and multi-turn conversation support. Includes Analytics Agent, Error Analyzer Agent, and optionally Code Intelligence Agent (`--enable-code-intelligence`). Available as `idp-cli chat --stack-name <stack>` for interactive use, `--prompt` flag for single-shot scripting, and `client.chat.send_message()` in the Python SDK. See `docs/idp-cli.md#chat`.
11+
1012
- **Multi-Document Discovery** — New capability to automatically discover document classes from a collection of documents. Instead of manually defining document schemas one at a time, users point to a folder of mixed documents and the system automatically identifies document types, clusters similar documents, generates JSON Schemas with field definitions for each type, and saves them to a configuration version — ready for immediate use in the processing pipeline. Available from the Web UI, CLI (`idp-cli discover-multidoc`), and SDK (`client.discovery.run_multi_doc()`).
1113
- **Web UI**: New "Multi-Document" tab on the Discovery page with job submission form (config version selector, bucket selector, S3 prefix input, zip upload), jobs table with search/filter/sort/pagination, and detailed job results page with pipeline progress, expandable JSON schemas, config deep-links, and Quality Review Report
1214
- **CLI**: `idp-cli discover-multidoc --dir ./samples/ -o ./schemas/` with Rich progress bars, results table, and reflection report

README.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -49,7 +49,7 @@ Concierge support for customization, deployment, and integration of production u
4949

5050
- **Serverless Architecture**: Built entirely on AWS serverless technologies including Lambda, Step Functions, SQS, and DynamoDB
5151
- **Modular, pluggable patterns**: Pre-built processing patterns using state-of-the-art models and AWS services
52-
- **Command Line Interface**: Programmatic batch processing with evaluation framework and analytics integration
52+
- **Command Line Interface**: Programmatic batch processing with evaluation framework, analytics integration, and interactive Agent Chat
5353
- **Advanced Classification**: Support for page-level and holistic document packet classification
5454
- **Few Shot Example Support**: Improve accuracy through example-based prompting
5555
- **Custom Business Logic Integration**: Inject custom prompt generation logic via Lambda functions for specialized document processing
@@ -174,7 +174,7 @@ For detailed deployment and testing instructions, see the [Deployment Guide](./d
174174
- [Architecture](./docs/architecture.md) - Detailed component architecture and data flow
175175
- [Demo Videos](./docs/demo-videos.md) - Comprehensive collection of feature demonstration videos
176176
- [Deployment](./docs/deployment.md) - Build, publish, deploy, and test instructions
177-
- [IDP CLI](./docs/idp-cli.md) - Command line interface for batch processing and evaluation workflows
177+
- [IDP CLI](./docs/idp-cli.md) - Command line interface for batch processing, evaluation workflows, and interactive Agent Chat
178178
- [Web UI](./docs/web-ui.md) - Web interface features and usage
179179
- [Agent Analysis](./docs/agent-analysis.md) - Natural language analytics and data visualization feature
180180
- [Custom MCP Agent](./docs/custom-MCP-agent.md) - Integrating external MCP servers for custom tools and capabilities

docs/agent-companion-chat.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,7 @@ The Agent Companion Chat provides an intelligent conversational interface throug
2424
- **Session Persistence**: Conversations are saved and resume where you left off
2525
- **Sample Prompts**: Pre-defined queries to help you get started quickly
2626
- **Privacy Controls**: Optional Code Intelligence with user consent for external services
27+
- **CLI & SDK Access**: Use the same agents from the terminal via `idp-cli chat` or programmatically via `IDPClient.chat` — see [IDP CLI chat command](./idp-cli.md#chat)
2728

2829
Updated Demo Video:
2930
https://github.com/user-attachments/assets/c48b9e48-e0d4-457c-8c95-8cdb7a9d332b

docs/idp-cli.md

Lines changed: 78 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,8 @@ A command-line tool for batch document processing with the GenAI IDP Accelerator
1414
📁 **Flexible Input** - Support for local files and S3 references
1515
🔍 **Comprehensive Status** - Track queued, running, completed, and failed documents
1616
📈 **Batch Analytics** - Success rates, durations, and detailed error reporting
17-
🎯 **Evaluation Framework** - Validate accuracy against baselines with detailed metrics
17+
🎯 **Evaluation Framework** - Validate accuracy against baselines with detailed metrics
18+
💬 **Agent Chat** - Interactive Agent Companion Chat from the terminal with Analytics, Error Analyzer, and more
1819

1920
Demo:
2021

@@ -49,6 +50,7 @@ https://github.com/user-attachments/assets/3d448a74-ba5b-4a4a-96ad-ec03ac0b4d7d
4950
- [config-list](#config-list)
5051
- [config-activate](#config-activate)
5152
- [config-delete](#config-delete)
53+
- [chat](#chat)
5254
- [Complete Evaluation Workflow](#complete-evaluation-workflow)
5355
- [Step 1: Deploy Your Stack](#step-1-deploy-your-stack)
5456
- [Step 2: Initial Processing from Local Directory](#step-2-initial-processing-from-local-directory)
@@ -2225,6 +2227,81 @@ idp-cli config-sync-bda --stack-name my-stack --config-version v2
22252227

22262228
---
22272229

2230+
### `chat`
2231+
2232+
Interactive Agent Companion Chat from the terminal. Provides access to the full multi-agent orchestrator including Analytics, Error Analyzer, Code Intelligence, and any configured External MCP Agents.
2233+
2234+
The chat command runs the same orchestrator as the Web UI's Agent Companion Chat, but locally in your terminal — with real-time streaming and multi-turn conversation support.
2235+
2236+
**Usage:**
2237+
```bash
2238+
idp-cli chat [OPTIONS]
2239+
```
2240+
2241+
**Options:**
2242+
- `--stack-name` (required): CloudFormation stack name
2243+
- `--region`: AWS region (optional)
2244+
- `--prompt`: Single-shot prompt — sends one message, prints the response, and exits. Useful for scripts and CI/CD.
2245+
- `--enable-code-intelligence`: Enable the Code Intelligence Agent (disabled by default because it uses external third-party services)
2246+
2247+
**Examples:**
2248+
2249+
```bash
2250+
# Interactive mode — multi-turn conversation
2251+
idp-cli chat --stack-name my-stack
2252+
2253+
# Single-shot mode — for scripts and automation
2254+
idp-cli chat --stack-name my-stack --prompt "What is the avg accuracy for the last test run?"
2255+
2256+
# With Code Intelligence enabled
2257+
idp-cli chat --stack-name my-stack --enable-code-intelligence
2258+
2259+
# Pipe output in scripts
2260+
idp-cli chat --stack-name my-stack --prompt "How many documents failed today?" 2>/dev/null
2261+
```
2262+
2263+
**Interactive session example:**
2264+
```
2265+
IDP Agent Chat
2266+
Stack: my-stack
2267+
2268+
✓ Ready Agents: Analytics Agent · Error Analyzer Agent · Code Intelligence Agent
2269+
Type /quit to exit
2270+
2271+
You: What is the avg accuracy for test run Fake-W2-Tax-Forms-20260320?
2272+
⟶ Analytics Agent
2273+
The average accuracy for test run Fake-W2-Tax-Forms-20260320 is 0.867 (86.7%) across 95 documents.
2274+
2275+
You: Break that down by document type
2276+
⟶ Analytics Agent
2277+
...
2278+
2279+
You: /quit
2280+
Goodbye.
2281+
```
2282+
2283+
**SDK usage:**
2284+
```python
2285+
from idp_sdk import IDPClient
2286+
2287+
client = IDPClient(stack_name="my-stack")
2288+
2289+
# Single message
2290+
resp = client.chat.send_message("How many documents were processed today?")
2291+
print(resp.response)
2292+
2293+
# Multi-turn conversation
2294+
resp2 = client.chat.send_message("Break down by type", session_id=resp.session_id)
2295+
print(resp2.response)
2296+
```
2297+
2298+
**Prerequisites:**
2299+
- Requires `idp_common[agents]` to be installed: `pip install -e 'lib/idp_common_pkg[agents]'`
2300+
- Requires Amazon Bedrock model access (Claude or Nova models)
2301+
- Stack must be deployed with Agent Companion Chat resources (DynamoDB tables, Athena database)
2302+
2303+
---
2304+
22282305
## Troubleshooting
22292306

22302307
### Stack Not Found

lib/idp_cli_pkg/idp_cli/chat.py

Lines changed: 149 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,149 @@
1+
# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
2+
# SPDX-License-Identifier: MIT-0
3+
4+
"""
5+
Chat command for idp-cli.
6+
7+
Runs the Agent Companion Chat orchestrator locally, providing interactive
8+
access to Analytics, Error Analyzer, and other agents from the terminal.
9+
"""
10+
11+
import asyncio
12+
import logging
13+
import re
14+
import uuid
15+
from typing import Optional
16+
17+
from rich.console import Console
18+
19+
logger = logging.getLogger(__name__)
20+
console = Console()
21+
22+
23+
def run_chat(
24+
stack_name: str,
25+
region: Optional[str] = None,
26+
prompt: Optional[str] = None,
27+
enable_code_intelligence: bool = False,
28+
):
29+
"""
30+
Run the chat command — interactive REPL or single-shot.
31+
32+
Args:
33+
stack_name: CloudFormation stack name
34+
region: AWS region
35+
prompt: If provided, run single-shot and exit
36+
"""
37+
console.print("[bold blue]IDP Agent Chat[/bold blue]")
38+
console.print(f"[dim]Stack: {stack_name}[/dim]")
39+
console.print()
40+
41+
# Use IDPClient to access chat processor internals
42+
from idp_sdk import IDPClient
43+
44+
# Suppress all logging — agents are very chatty
45+
logging.disable(logging.CRITICAL)
46+
47+
try:
48+
client = IDPClient(stack_name=stack_name, region=region)
49+
processor = client.chat._get_processor()
50+
51+
with console.status("[bold]Discovering stack resources..."):
52+
processor._setup_env()
53+
54+
session_id = f"cli-{uuid.uuid4().hex[:12]}"
55+
56+
with console.status("[bold]Initializing agents..."):
57+
processor._ensure_orchestrator(
58+
session_id,
59+
enable_code_intelligence=enable_code_intelligence,
60+
)
61+
62+
# Show available agents
63+
from idp_common.agents.factory import agent_factory
64+
65+
agents = agent_factory.list_available_agents()
66+
agent_names = [a["agent_name"] for a in agents]
67+
console.print(
68+
f"[green]✓ Ready[/green] [dim]Agents: {' · '.join(agent_names)}[/dim]"
69+
)
70+
console.print("[dim]Type /quit to exit[/dim]\n")
71+
72+
if prompt:
73+
_handle_prompt(processor._orchestrator, prompt)
74+
return
75+
76+
# Interactive REPL — reuse a single event loop for the session
77+
loop = asyncio.new_event_loop()
78+
try:
79+
while True:
80+
try:
81+
user_input = console.input("[bold cyan]You:[/bold cyan] ")
82+
except (EOFError, KeyboardInterrupt):
83+
console.print("\n[dim]Goodbye.[/dim]")
84+
break
85+
86+
text = user_input.strip()
87+
if not text:
88+
continue
89+
if text.lower() in ("/quit", "/exit", "quit", "exit"):
90+
console.print("[dim]Goodbye.[/dim]")
91+
break
92+
93+
_handle_prompt(processor._orchestrator, text, loop=loop)
94+
finally:
95+
loop.close()
96+
finally:
97+
# Restore logging so callers aren't permanently silenced
98+
logging.disable(logging.NOTSET)
99+
100+
101+
def _handle_prompt(orchestrator, prompt: str, loop=None):
102+
"""Send a prompt to the orchestrator and stream the response."""
103+
console.print()
104+
105+
if loop is not None:
106+
# Reuse persistent loop (interactive REPL)
107+
try:
108+
loop.run_until_complete(_stream_response(orchestrator, prompt))
109+
except Exception as e:
110+
console.print(f"[red]Error: {e}[/red]")
111+
else:
112+
# One-shot mode — create and tear down a loop
113+
one_shot = asyncio.new_event_loop()
114+
try:
115+
one_shot.run_until_complete(_stream_response(orchestrator, prompt))
116+
except Exception as e:
117+
console.print(f"[red]Error: {e}[/red]")
118+
finally:
119+
one_shot.close()
120+
121+
console.print()
122+
123+
124+
async def _stream_response(orchestrator, prompt: str) -> str:
125+
"""Stream orchestrator response, printing chunks as they arrive."""
126+
full_text = ""
127+
displayed = ""
128+
current_subagent = None
129+
130+
async for event in orchestrator.stream_async(prompt):
131+
if "data" in event:
132+
full_text += event["data"]
133+
clean = re.sub(
134+
r"<thinking>.*?</thinking>", "", full_text, flags=re.DOTALL
135+
).strip()
136+
if len(clean) > len(displayed):
137+
new = clean[len(displayed) :]
138+
console.print(new, end="", highlight=False)
139+
displayed = clean
140+
141+
elif "current_tool_use" in event:
142+
tool_name = event["current_tool_use"].get("name", "")
143+
if tool_name and tool_name != current_subagent:
144+
current_subagent = tool_name
145+
display_name = tool_name.replace("_agent", "").replace("_", " ").title()
146+
console.print(f"\n[dim]⟶ {display_name}[/dim]", highlight=False)
147+
148+
console.print()
149+
return displayed

lib/idp_cli_pkg/idp_cli/cli.py

Lines changed: 62 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@
1111
import os
1212
import sys
1313
import time
14-
from typing import Optional
14+
from typing import TYPE_CHECKING, Optional
1515

1616
_SETUP_HELP = """\
1717
Error: Required packages not found.
@@ -40,13 +40,14 @@
4040
except ImportError:
4141
# Dependencies not installed — main() will print a helpful message and exit.
4242
# Define minimal stubs so the module can still be imported for entry point resolution.
43-
click = None # type: ignore
44-
IDPClient = None # type: ignore
45-
Console = None # type: ignore
46-
Live = None # type: ignore
47-
Table = None # type: ignore
48-
display = None # type: ignore
49-
boto3 = None # type: ignore
43+
if not TYPE_CHECKING:
44+
click = None
45+
IDPClient = None
46+
Console = None
47+
Live = None
48+
Table = None
49+
display = None
50+
boto3 = None
5051

5152
# Configure logging
5253
logging.basicConfig(
@@ -5265,6 +5266,59 @@ def publish(
52655266
sys.exit(1)
52665267

52675268

5269+
@cli.command()
5270+
@click.option("--stack-name", required=True, help="CloudFormation stack name")
5271+
@click.option("--region", default=None, help="AWS region")
5272+
@click.option(
5273+
"--prompt",
5274+
default=None,
5275+
help="Single-shot prompt (non-interactive mode)",
5276+
)
5277+
@click.option(
5278+
"--enable-code-intelligence",
5279+
is_flag=True,
5280+
default=False,
5281+
help="Enable Code Intelligence Agent (uses external third-party services)",
5282+
)
5283+
def chat(
5284+
stack_name: str,
5285+
region: Optional[str],
5286+
prompt: Optional[str],
5287+
enable_code_intelligence: bool,
5288+
):
5289+
"""Interactive Agent Companion Chat from the terminal.
5290+
5291+
Provides access to the full multi-agent orchestrator including
5292+
Analytics, Error Analyzer, and other agents.
5293+
5294+
\b
5295+
Examples:
5296+
# Interactive mode
5297+
idp-cli chat --stack-name IDP
5298+
5299+
# Single-shot mode
5300+
idp-cli chat --stack-name IDP --prompt "How many documents were processed today?"
5301+
5302+
# With Code Intelligence (external services)
5303+
idp-cli chat --stack-name IDP --enable-code-intelligence
5304+
"""
5305+
try:
5306+
from .chat import run_chat
5307+
except ImportError:
5308+
console.print(
5309+
"[red]✗ Chat requires idp_common[agents] to be installed.\n"
5310+
" Run: pip install -e 'lib/idp_common_pkg[agents]'[/red]"
5311+
)
5312+
sys.exit(1)
5313+
5314+
run_chat(
5315+
stack_name=stack_name,
5316+
region=region,
5317+
prompt=prompt,
5318+
enable_code_intelligence=enable_code_intelligence,
5319+
)
5320+
5321+
52685322
def main():
52695323
"""Main entry point for the CLI"""
52705324
# Pre-flight check: verify core dependencies are importable

0 commit comments

Comments
 (0)