Skip to content

Commit e30e2fb

Browse files
abossardCopilot
andcommitted
feat: add SSE activity monitor, settings page, agent templates & run history
- Agent Activity page with real-time SSE event stream (tool calls, LLM thinking, run lifecycle), filterable by run_id via URL query param - EventBus pub/sub + StreamingCallbackHandler wired into ReAct engine - Settings page: drag-and-drop tab reorder, hide/show toggles, icon picker (57 FluentUI icons), persisted to localStorage - Agent templates dropdown (KBA from tickets, worklog stats, next step advisor) pre-fills the create agent form - AgentRunPage now shows filtered run history with detail dialog and link to Activity page filtered by run_id - 19 new Playwright E2E tests (8 activity + 11 settings) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1 parent 2c36518 commit e30e2fb

17 files changed

Lines changed: 2223 additions & 59 deletions

File tree

CLAUDE.md

Lines changed: 129 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,129 @@
1+
# CLAUDE.md
2+
3+
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
4+
5+
## Commands
6+
7+
### Setup
8+
```bash
9+
./setup.sh # Bootstrap: creates .venv, installs deps, installs Playwright
10+
source .venv/bin/activate
11+
```
12+
13+
### Run (Development)
14+
```bash
15+
./start-dev.sh # Starts backend (port 5001) + frontend (port 3001) together
16+
17+
# Or manually:
18+
# Terminal 1 - Backend:
19+
source .venv/bin/activate && cd backend && python app.py
20+
21+
# Terminal 2 - Frontend:
22+
cd frontend && npm run dev
23+
```
24+
25+
### Run (Production / Docker)
26+
```bash
27+
docker build -t quart-react-demo .
28+
docker run -p 5001:5001 quart-react-demo
29+
# Quart serves both API + built frontend on port 5001
30+
```
31+
32+
### Tests
33+
```bash
34+
# E2E tests (Playwright - requires both servers running or auto-started)
35+
npm run test:e2e # Run all E2E tests (chromium, firefox, webkit)
36+
npm run test:e2e:ui # Interactive UI mode
37+
npm run test:e2e:report # View last HTML report
38+
npx playwright test tests/e2e/workbench.spec.js --project=chromium # Single file + browser
39+
40+
# Backend unit tests (pytest)
41+
cd backend
42+
pytest # All backend tests
43+
pytest agent_builder/tests/ # 132 agent framework tests
44+
pytest test_agents.py -v # Single file, verbose
45+
```
46+
47+
### Frontend Build
48+
```bash
49+
cd frontend && npm install && npm run build # Outputs to frontend/dist/
50+
```
51+
52+
## Architecture
53+
54+
### Overview
55+
Full-stack teaching app: **Quart** (async Python ASGI, port 5001) + **React/Vite** (port 3001 in dev). In development, Vite proxies `/api/*` to the Quart backend. In production, Quart serves the built frontend statically.
56+
57+
### Backend (`backend/`)
58+
59+
**Key pattern — Unified Operation System** (`api_decorators.py`):
60+
```python
61+
@operation("create_task", "Create a task", http_method="POST", mcp_enabled=True)
62+
def op_create_task(data: TaskCreate) -> Task:
63+
return TaskService.create_task(data)
64+
```
65+
The `@operation` decorator auto-generates: REST endpoint, MCP JSON-RPC method, JSON schema, and LangChain `StructuredTool`. Define once, used everywhere. Operations are registered in `operations.py`.
66+
67+
**Deep module pattern**: Business logic consolidated into service classes (`TaskService`, `WorkbenchService`, `CSVTicketService`, `KBAService`) rather than scattered thin functions.
68+
69+
**Key files:**
70+
- `app.py` — Quart app, route/blueprint registration, SSE streams, LLM config
71+
- `api_decorators.py``@operation` decorator implementation
72+
- `operations.py` — All operation definitions
73+
- `tasks.py` — SQLModel task models + `TaskService` (SQLite via `data/tasks.db`)
74+
- `csv_data.py``CSVTicketService`: loads CSV (`CSV/data.csv`), date parsing (German format `DD.MM.YYYY HH:MM:SS`), filtering, pagination
75+
- `tickets.py` — Pydantic ticket models + enums (Status, Priority, Urgency, Impact)
76+
- `agent_builder/` — Full agent lifecycle: `service.py` (WorkbenchService), `engine/` (ReAct loop), `persistence/` (SQLite repos), `tools/` (ToolRegistry, MCP adapters), `routes.py` (Blueprint at `/api/workbench/*`)
77+
- `kba_service.py` — Knowledge Base Article generation pipeline
78+
79+
**LLM config**: LiteLLM is default (works with GitHub Copilot, no key needed); OpenAI is optional override. Config via `.env`.
80+
81+
**MCP**: Backend exposes both REST and MCP JSON-RPC from the same `@operation` handlers. `mcp_handler.py` routes JSON-RPC to operations.
82+
83+
### Frontend (`frontend/src/`)
84+
85+
**Feature-first structure**: `features/<name>/` contains page component, subcomponents, utils, and local state. All backend calls go through `services/api.js`.
86+
87+
**Key features:**
88+
- `workbench/` — Agent Fabric: create/run/inspect ReAct agents with live schema editor
89+
- `csvtickets/` — Browse/filter/sort/paginate the CSV ticket dataset
90+
- `tickets/` — Nivo chart visualizations (bar, sankey, stream)
91+
- `usecase-demo/` — Pre-built demo agent runs with custom result renderers
92+
- `kba-drafter/` — KBA generation UI (draft list, editor, audit trail)
93+
- `dashboard/` — Real-time updates via Server-Sent Events (`/api/time-stream`)
94+
- `agent/` — One-shot agent chat interface
95+
96+
**UI**: FluentUI v9 (`@fluentui/react-components`) throughout. Charts via `@nivo/*`.
97+
98+
### Testing
99+
100+
E2E tests (`tests/e2e/`) use Playwright. `playwright.config.js` auto-starts backend + frontend via `webServer`. Tests use `data-testid` attributes. Three browser projects: chromium, firefox, webkit.
101+
102+
Backend tests use pytest. `agent_builder/tests/` has 132 tests covering the agent framework.
103+
104+
## Environment
105+
106+
Copy `.env.example` to `.env`. Key variables:
107+
```
108+
# LiteLLM (default — no key required if using GitHub Copilot)
109+
LITELLM_MODEL=github_copilot/gpt-4o
110+
111+
# OpenAI (optional override)
112+
OPENAI_API_KEY=sk-proj-...
113+
OPENAI_MODEL=gpt-4o-mini
114+
115+
# KBA publishing path
116+
KB_FILE_BASE_PATH=./kb_published
117+
```
118+
119+
## Ports
120+
- Backend (Quart): `http://localhost:5001`
121+
- Frontend dev (Vite): `http://localhost:3001`
122+
- In production: single port `5001` (Quart serves everything)
123+
124+
## Docs
125+
- `docs/QUICKSTART.md` — 5-minute setup
126+
- `docs/AGENT_BUILDER.md` — Agent framework architecture with diagrams
127+
- `docs/UNIFIED_ARCHITECTURE.md` — REST + MCP integration patterns
128+
- `docs/LEARNING.md` — Design principles (Grokking Simplicity, deep modules)
129+
- `docs/TROUBLESHOOTING.md` — Common issues

backend/agent_builder/engine/__init__.py

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,11 @@
11
"""
22
Agent Builder — Engine
33
4-
ReAct agent execution engine: runner, callbacks, prompt building.
4+
ReAct agent execution engine: runner, callbacks, prompt building, event bus.
55
"""
66

7-
from .callbacks import make_llm_logging_callback, make_tool_logging_callback
7+
from .callbacks import make_llm_logging_callback, make_streaming_callback, make_tool_logging_callback
8+
from .event_bus import AgentEvent, AgentEventBus, agent_event_bus
89
from .prompt_builder import (
910
DEFAULT_OUTPUT_SCHEMA,
1011
append_markdown_instruction,
@@ -15,8 +16,11 @@
1516
from .react_runner import RunResult, build_llm, build_react_agent, extract_tools_used, run_react_agent
1617

1718
__all__ = [
19+
"AgentEvent",
20+
"AgentEventBus",
1821
"DEFAULT_OUTPUT_SCHEMA",
1922
"RunResult",
23+
"agent_event_bus",
2024
"append_markdown_instruction",
2125
"append_output_instructions",
2226
"build_chat_system_prompt",
@@ -25,6 +29,7 @@
2529
"build_react_agent",
2630
"extract_tools_used",
2731
"make_llm_logging_callback",
32+
"make_streaming_callback",
2833
"make_tool_logging_callback",
2934
"run_react_agent",
3035
]

backend/agent_builder/engine/callbacks.py

Lines changed: 103 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,14 +2,16 @@
22
Agent Builder — LLM Callbacks
33
44
Logging callbacks for OpenAI LLM calls and tool invocations.
5-
Actions (I/O): write to logger.
5+
Actions (I/O): write to logger and event bus.
66
"""
77

88
import logging
99
from time import perf_counter
1010
from typing import Any
1111
from uuid import UUID
1212

13+
from .event_bus import AgentEvent, AgentEventBus
14+
1315
logger = logging.getLogger(__name__)
1416

1517

@@ -148,3 +150,103 @@ def _extract_llm_call_metadata(
148150
finish_reason = finish_reason or maybe_finish
149151

150152
return token_usage, model_name, finish_reason
153+
154+
155+
# ---------------------------------------------------------------------------
156+
# SSE Streaming Callback — publishes events to EventBus
157+
# ---------------------------------------------------------------------------
158+
159+
def make_streaming_callback(run_id: str, event_bus: AgentEventBus) -> Any:
160+
"""Create a callback handler that publishes agent events to the SSE event bus."""
161+
from langchain_core.callbacks import BaseCallbackHandler
162+
163+
class StreamingCallbackHandler(BaseCallbackHandler):
164+
def __init__(self) -> None:
165+
super().__init__()
166+
self._start_times: dict[Any, float] = {}
167+
168+
def on_tool_start(
169+
self,
170+
serialized: dict[str, Any],
171+
input_str: str,
172+
*,
173+
run_id: Any = None,
174+
**kwargs: Any,
175+
) -> None:
176+
self._start_times[run_id] = perf_counter()
177+
name = serialized.get("name", "unknown")
178+
preview = input_str[:500] if isinstance(input_str, str) else str(input_str)[:500]
179+
event_bus.publish(AgentEvent(
180+
run_id=run_id_outer,
181+
event_type="tool_start",
182+
data={"tool_name": name, "input": preview},
183+
))
184+
185+
def on_tool_end(self, output: str, *, run_id: Any = None, **kwargs: Any) -> None:
186+
started = self._start_times.pop(run_id, None)
187+
duration_ms = int((perf_counter() - started) * 1000) if started is not None else None
188+
preview = output[:500] if isinstance(output, str) else str(output)[:500]
189+
event_bus.publish(AgentEvent(
190+
run_id=run_id_outer,
191+
event_type="tool_end",
192+
data={"tool_name": kwargs.get("name", ""), "output": preview, "duration_ms": duration_ms},
193+
))
194+
195+
def on_tool_error(self, error: BaseException, *, run_id: Any = None, **kwargs: Any) -> None:
196+
started = self._start_times.pop(run_id, None)
197+
duration_ms = int((perf_counter() - started) * 1000) if started is not None else None
198+
event_bus.publish(AgentEvent(
199+
run_id=run_id_outer,
200+
event_type="tool_error",
201+
data={"error": str(error), "duration_ms": duration_ms},
202+
))
203+
204+
def on_llm_start(
205+
self,
206+
serialized: dict[str, Any],
207+
prompts: list[str],
208+
*,
209+
run_id: UUID = None,
210+
**kwargs: Any,
211+
) -> None:
212+
self._start_times[run_id] = perf_counter()
213+
model_name = None
214+
if isinstance(serialized, dict):
215+
model_name = (
216+
serialized.get("kwargs", {}).get("model")
217+
if isinstance(serialized.get("kwargs"), dict)
218+
else None
219+
)
220+
event_bus.publish(AgentEvent(
221+
run_id=run_id_outer,
222+
event_type="llm_start",
223+
data={"model": model_name or ""},
224+
))
225+
226+
def on_llm_end(self, response: Any, *, run_id: UUID = None, **kwargs: Any) -> None:
227+
started_at = self._start_times.pop(run_id, None)
228+
duration_ms = int((perf_counter() - started_at) * 1000) if started_at is not None else None
229+
token_usage, model_name, finish_reason = _extract_llm_call_metadata(response)
230+
event_bus.publish(AgentEvent(
231+
run_id=run_id_outer,
232+
event_type="llm_end",
233+
data={
234+
"model": model_name or "",
235+
"duration_ms": duration_ms,
236+
"token_usage": token_usage or {},
237+
"finish_reason": finish_reason or "",
238+
},
239+
))
240+
241+
def on_llm_error(self, error: BaseException, *, run_id: UUID = None, **kwargs: Any) -> None:
242+
started_at = self._start_times.pop(run_id, None)
243+
duration_ms = int((perf_counter() - started_at) * 1000) if started_at is not None else None
244+
event_bus.publish(AgentEvent(
245+
run_id=run_id_outer,
246+
event_type="llm_error",
247+
data={"error": str(error), "duration_ms": duration_ms},
248+
))
249+
250+
# Closure captures run_id as run_id_outer to avoid shadowing with LangChain's run_id param
251+
run_id_outer = run_id
252+
return StreamingCallbackHandler()
Lines changed: 77 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,77 @@
1+
"""
2+
Agent Builder — Event Bus
3+
4+
Pub/sub for real-time agent activity events.
5+
Subscribers receive events via asyncio.Queue; a history buffer
6+
provides catch-up for new SSE connections.
7+
8+
Data: AgentEvent dataclass
9+
Action: publish / subscribe / unsubscribe (I/O via queues)
10+
"""
11+
12+
import asyncio
13+
import logging
14+
import time
15+
from dataclasses import asdict, dataclass, field
16+
from typing import Any
17+
18+
logger = logging.getLogger(__name__)
19+
20+
MAX_HISTORY = 200
21+
22+
23+
@dataclass
24+
class AgentEvent:
25+
"""A single event from an agent run."""
26+
run_id: str
27+
event_type: str
28+
data: dict[str, Any] = field(default_factory=dict)
29+
timestamp: float = field(default_factory=time.time)
30+
31+
def to_sse_dict(self) -> dict[str, Any]:
32+
return asdict(self)
33+
34+
35+
class AgentEventBus:
36+
"""Simple in-process pub/sub for agent execution events."""
37+
38+
def __init__(self, max_history: int = MAX_HISTORY) -> None:
39+
self._subscribers: list[asyncio.Queue[AgentEvent]] = []
40+
self._history: list[AgentEvent] = []
41+
self._max_history = max_history
42+
43+
def subscribe(self) -> asyncio.Queue[AgentEvent]:
44+
q: asyncio.Queue[AgentEvent] = asyncio.Queue(maxsize=500)
45+
self._subscribers.append(q)
46+
logger.debug("EventBus: subscriber added (total=%d)", len(self._subscribers))
47+
return q
48+
49+
def unsubscribe(self, q: asyncio.Queue[AgentEvent]) -> None:
50+
try:
51+
self._subscribers.remove(q)
52+
logger.debug("EventBus: subscriber removed (total=%d)", len(self._subscribers))
53+
except ValueError:
54+
pass
55+
56+
def publish(self, event: AgentEvent) -> None:
57+
"""Publish an event to all subscribers and history buffer.
58+
59+
Synchronous — safe to call from LangChain BaseCallbackHandler methods
60+
because asyncio.Queue.put_nowait() doesn't require awaiting.
61+
"""
62+
self._history.append(event)
63+
if len(self._history) > self._max_history:
64+
self._history = self._history[-self._max_history:]
65+
66+
for q in self._subscribers:
67+
try:
68+
q.put_nowait(event)
69+
except asyncio.QueueFull:
70+
logger.warning("EventBus: subscriber queue full, dropping event")
71+
72+
def get_history(self) -> list[AgentEvent]:
73+
return list(self._history)
74+
75+
76+
# Global singleton
77+
agent_event_bus = AgentEventBus()

0 commit comments

Comments
 (0)