Commit 1416147
Agent workbench v2 (#19)
* Extract agent builder into extensible module with tests
Create backend/agent_builder/ as a standalone, deeply layered module
following Grokking Simplicity (data/calculations/actions separation)
and A Philosophy of Software Design (deep modules).
Structure:
- models/: Pure data (Pydantic/SQLModel) - agent, run, evaluation, chat
- tools/: ToolRegistry, schema converter, MCP adapter
- engine/: Unified ReAct runner, callbacks, prompt builder
- evaluator.py: Success criteria evaluation (mostly calculations)
- persistence/: DB engine setup + repository pattern
- service.py: WorkbenchService (deep module facade)
- chat_service.py: ChatService using shared ReAct engine
- routes.py: Quart Blueprint replacing 200+ lines from app.py
- tests/: 107 tests (unit + integration + E2E)
Key improvements:
- Eliminated duplicate ReAct agent building (was in both agents.py
and agent_workbench/service.py)
- DRY error handling in routes via Blueprint
- Repository pattern isolates DB from business logic
- Pure calculation modules (prompt_builder, schema_converter,
evaluator) are independently testable
- Backward-compatible: agent_workbench/__init__.py shims to new module
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Add per-agent LLM config: model, temperature, recursion_limit, max_tokens, output_instructions
Each AgentDefinition now stores configurable LLM parameters:
- model: override service default (e.g. gpt-4o vs gpt-4o-mini)
- temperature: 0.0-2.0 (deterministic to creative)
- recursion_limit: 1-100 max ReAct loop iterations
- max_tokens: cap response length (0 = unlimited)
- output_instructions: custom formatting (replaces default markdown)
Changes:
- models/agent.py: 5 new fields with validation (ge/le bounds)
- persistence/database.py: migrations for existing DBs
- engine/react_runner.py: build_llm accepts temperature+max_tokens
- engine/prompt_builder.py: append_output_instructions for custom formatting
- service.py: _resolve_llm_for_agent builds per-agent LLM when config differs
- routes.py: ui-config v2 exposes llm_config_fields and defaults
- 12 new tests (model validation, CRUD, E2E roundtrip via REST)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Add output_schema for type-safe structured output, fix defaults
Changes:
- recursion_limit default: 10 → 3 (most agents finish in 1-3 tool calls)
- max_tokens default: 0 → 4096 (sensible cap instead of unlimited)
- New field: output_schema (JSON Schema stored as JSON in DB)
output_schema is config, not code. You define the expected response
shape as a JSON Schema:
{"type":"object","properties":{"breaches":{"type":"array",...}}}
At runtime this does two things:
1. Injected into system prompt so the LLM knows the expected structure
2. Takes priority over output_instructions and default markdown
Priority chain for output formatting:
output_schema (strict JSON) > output_instructions (free text) > default markdown
128 tests pass (9 new tests for schema handling).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Add suggest-schema endpoint and UI button
New endpoint: POST /api/workbench/suggest-schema
Takes agent name, description, system_prompt and asks the LLM to
propose a JSON Schema for the agent's structured output.
Backend:
- service.py: suggest_schema() method - builds a prompt, calls LLM,
parses JSON response (handles markdown fences), falls back to
generic schema on parse failure
- routes.py: POST /api/workbench/suggest-schema route
Frontend:
- api.js: suggestOutputSchema() function
- WorkbenchPage.jsx: output schema textarea + Suggest Schema button
in the create form. Schema is editable JSON, sent as output_schema
on agent creation. Button disabled until name or prompt is filled.
129 tests pass (1 new E2E test for suggest-schema endpoint).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Wire output_schema to LangGraph response_format for SDK-level enforcement
When an agent has output_schema configured, it now does TWO things:
1. Prompt injection (existing) — schema is described in the system prompt
so the LLM understands the expected structure
2. SDK enforcement (new) — schema is passed as response_format to
create_react_agent(), which uses LangGraph's built-in structured
output mechanism (provider-native or tool-based)
At runtime, structured_response from the LangGraph result takes
priority over raw message content. If the agent has no output_schema,
behavior is unchanged (markdown output from final message).
The output pipeline:
output_schema defined → response_format=schema → structured_response → JSON
no output_schema → final message content → markdown (default)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Always use structured_response with default schema
Every agent now always returns structured output via LangGraph's
response_format — no more untyped markdown strings.
Default schema (when no custom output_schema is set):
{
"message": "string (markdown)",
"referenced_tickets": ["string"]
}
This means:
- Plain agents → get {message: '...markdown...', referenced_tickets: [...]}
- Custom schema agents → get whatever schema they define
- Both enforced at SDK level via response_format, not just prompt
Changes:
- prompt_builder.py: DEFAULT_OUTPUT_SCHEMA, resolve_output_schema()
- service.py: always passes effective schema to create_react_agent
- routes.py: ui-config exposes default_output_schema for frontend
- Tests updated (132 pass)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Add comprehensive docs with mermaid diagrams, clean up stale docs
New: docs/AGENT_BUILDER.md — full architecture documentation with:
- Architecture diagram (module layers + data flow)
- Sequence diagram (agent run lifecycle)
- Structured output pipeline flowchart
- ER diagram (DB schema)
- Data/Calculations/Actions separation diagram
- Deep modules table
- Extensibility flowchart
- API endpoint reference
- Testing commands
Updated:
- AGENTS_IMPLEMENTATION.md — replaced stale content with summary + pointer
- docs/AGENTS.md — replaced stale architecture with mermaid + pointer
- docs/PROJECT_STRUCTURE.md — added agent_builder/ to tree
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Docs overhaul + remove ~1800 lines of dead code/stale docs
Documentation:
- README.md: Complete rewrite with features table, screenshots, mermaid
architecture diagram, agent builder section, correct tech stack
- PROJECT_STRUCTURE.md: Full rewrite matching actual codebase
- AGENTS.md: Fixed AgentService→WorkbenchService, updated examples
- LEARNING.md: Fixed broken link
Deleted stale docs:
- AGENTS_IMPLEMENTATION.md (was a 3-line redirect stub)
- docs/RULES.md (empty file)
- docs/SQLMODEL_MIGRATION.md (historical, migration complete)
Dead code removed from agents.py (~250 lines):
- MCP client stubs (_mcp_tool_to_langchain, _ensure_ticket_mcp_connection, close)
- Schema helpers only used by dead MCP code (_json_type_to_python, _schema_to_pydantic)
- OpenAI logging callback (duplicated in agent_builder/engine/callbacks.py)
- _build_state_graph learning example (dead code)
- Unused imports (get_langchain_tools, MCPClient, create_model)
Deleted old agent_workbench/ source files (~1030 lines):
- models.py, service.py, evaluator.py, tool_registry.py
- Only __init__.py shim remains for backward compatibility
132 backend tests + 15 Playwright tests pass.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Add Playwright tests for suggest-schema and agent chat
New E2E tests in workbench.spec.js:
- 'creates agent with output schema via suggest button' — mocks
/api/workbench/suggest-schema, clicks Suggest Schema, verifies
schema populates textarea, creates agent, deletes it
- 'sends message and displays mocked response' (Agent Chat UI) —
mocks /api/agents/run, types message, clicks send, verifies
markdown heading and tool badge render
17 Playwright tests pass (was 15, +2 new).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Add VPN agent and failure handling Playwright tests
New Agent Fabric E2E tests:
- 'runs VPN troubleshooting agent and verifies structured output'
Creates agent with VPN analysis prompt, runs it (mocked),
verifies structured JSON output with ticket IDs (INC-101, INC-312),
referenced_tickets field, and VPN content in rendered output
- 'handles agent run failure gracefully'
Creates agent, runs it with mocked failure response,
verifies UI doesn't crash and shows completion state
19 Playwright tests pass (was 17, +2 new).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix structured output rendering in Agent Fabric UI
The output is now always structured JSON ({message, referenced_tickets}).
The UI now parses it and renders each part appropriately:
- message → rendered as GitHub-flavored Markdown (ReactMarkdown)
- referenced_tickets → rendered as monospace badges below the output
- Extra custom schema fields → rendered as formatted JSON in a pre block
- Button preview → shows message text, not raw JSON
Also handles non-JSON output gracefully (falls back to raw markdown).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Add MCP App technical documentation
New: docs/MCP_APP.md — comprehensive guide on how this project
works as an MCP application:
- What an MCP App is (app that exposes business logic via MCP protocol)
- Architecture diagrams: consumers (Claude, Copilot, agents) → MCP endpoint
- Full protocol sequence diagram (initialize → tools/list → tools/call)
- The @operation decorator: single source of truth for REST + MCP + LangChain
- How to connect clients (Claude Desktop, Python, curl examples)
- 4-layer architecture diagram (business logic → operations → adapters → consumers)
- Extension roadmap: Resources, Prompts, SSE streaming
- Security considerations table
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Add SchemaRenderer + visual SchemaEditor with x-ui widget system
SchemaRenderer (frontend/src/features/workbench/SchemaRenderer.jsx):
- Generic component: takes {data, schema} and renders each property
using x-ui widget annotations
- Widgets: markdown, table, badge-list, stat-card, bar-chart (Nivo),
pie-chart (Nivo), json, hidden
- Auto-detection when no x-ui: string→markdown, integer→stat-card,
array of objects→table, array of strings→badge-list, object→json
- Console debug logging, data-testid per field for E2E testing
SchemaEditor (frontend/src/features/workbench/SchemaEditor.jsx):
- Visual property list editor (no raw JSON editing needed)
- Add/remove properties, set name/type/description
- Widget picker dropdown with all available widgets
- Context-sensitive options (columns for table, label for stat-card,
indexBy/keys for bar-chart)
- Syncs with suggest-schema: LLM suggestion populates visual editor
- Outputs valid JSON Schema with x-ui annotations
Backend:
- DEFAULT_OUTPUT_SCHEMA now has x-ui annotations (markdown + badge-list)
- suggest_schema prompt updated to suggest x-ui widgets per property
Wiring:
- WorkbenchPage uses SchemaRenderer for run output (replaces hardcoded)
- WorkbenchPage uses SchemaEditor for create form (replaces textarea)
20 Playwright tests pass (including new SchemaRenderer widget test).
132 backend tests pass.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Improve suggest-schema prompt with full data domain + widget docs
The suggest-schema LLM prompt now includes:
- Ticket data domain (all field names, types, enum values, example cities)
- Available tools with descriptions (csv_list_tickets, csv_search_tickets, etc.)
- Full widget documentation with use-cases and options for each:
markdown, table (columns), badge-list, stat-card (label),
bar-chart (indexBy, keys), pie-chart, json, hidden
- Explicit rules: always include message+referenced_tickets,
match widget to data shape, use snake_case names
This gives the LLM enough context to suggest schemas that actually
match the ticket data (e.g. status distribution → pie-chart,
ticket list → table with incident_id/summary/status columns).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix latency issues: schema title bug + recursion_limit headroom
Investigation found 3 root causes for slow AI calls:
1. gpt-5-nano is a REASONING model — burns 192-832 reasoning tokens
per LLM call (invisible chain-of-thought), taking 2-8s each.
A simple 'say hello' costs 8.4s with 832 reasoning tokens.
2. response_format adds a 3rd LLM call — LangGraph's
generate_structured_response makes a separate LLM call to format
the output as JSON after the ReAct loop finishes.
Without: 4.7s (2 calls). With: 13s (3 calls).
3. Missing 'title' in output_schema crashed with_structured_output.
OpenAI's API requires a top-level 'title' in the JSON Schema.
Fixes applied:
- resolve_output_schema() now auto-adds 'title': 'AgentOutput'
when missing (both default and custom schemas)
- DEFAULT_OUTPUT_SCHEMA has explicit 'title' field
- recursion_limit: user's setting (default 3) is now multiplied by 4
for the actual LangGraph graph, with a floor of 10. This prevents
GraphRecursionError when response_format adds extra graph steps.
Note: The main latency driver (reasoning tokens) is inherent to the
model choice. Users can switch to gpt-4o-mini via per-agent 'model'
field for ~10x faster non-reasoning responses.
133 backend + 20 Playwright tests pass.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix agent tool token bloat: compact fields + lower default limits
Root cause: csv_list_tickets tool returned full Ticket objects with ALL
fields (notes, description, resolution, work logs) — ~65K tokens for
100 tickets. The LLM had to process all of this, causing 30-60s per
step with a reasoning model.
Changes to operations.py:
- csv_list_tickets: returns compact dicts (10 fields, not 30+),
default limit 25 (was 100), max limit 100 (was 500)
- csv_search_tickets: same compact treatment, limit 25 (was 50)
- csv_get_ticket: now accepts optional 'fields' parameter for
selective detail drill-down, returns dict (was full Ticket)
- Tool descriptions updated to guide agents: 'use csv_get_ticket
for full details' pattern
Token impact per tool call:
Before: 100 tickets × ~400 tokens = ~65,000 tokens
After: 25 tickets × ~60 tokens = ~1,500 tokens (97% reduction)
Expected latency improvement:
Before: ~13s per tool call (65K token input processing)
After: ~3-5s per tool call (1.5K token input)
153 tests pass (133 backend + 20 Playwright).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Drop response_format to eliminate extra LLM call
LangGraph 1.0.8 implements response_format via a SEPARATE LLM call
(generate_structured_response) — adding 5-10s latency per run.
The refactor to inline tool-based structured output (github.com/
langchain-ai/langgraph/issues/5872) hasn't shipped yet.
Fix: remove response_format from create_react_agent. The system
prompt already instructs the LLM to produce JSON matching the
schema (via append_output_instructions). The frontend's
SchemaRenderer handles both parsed JSON and raw text gracefully.
Latency impact:
Before: 3 LLM calls (decide tool + answer + format JSON) ~13s
After: 2 LLM calls (decide tool + answer as JSON) ~5s
When LangGraph ships inline structured output, we can re-enable
response_format with zero code changes (just pass it back to
build_react_agent).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Enable OpenAI JSON mode for guaranteed valid JSON output
Adds response_format: {type: 'json_object'} to the ChatOpenAI
constructor via model_kwargs. This is a model-level setting that
constrains token generation to valid JSON — no extra LLM call,
no post-processing, just guaranteed JSON from every response.
This is different from LangGraph's response_format parameter
(which adds a separate LLM call). This is OpenAI's native JSON
mode applied at the API level during the same call.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Revert JSON mode — incompatible with non-strict tool schemas
OpenAI's response_format: json_object requires all tools to have
strict schemas. Our tools (from @operation decorator) don't set
strict=True, causing: 'csv_search_tickets is not strict. Only
strict function tools can be auto-parsed'.
Reverting to prompt-only JSON enforcement, which tested at 3/3
reliability with gpt-5-nano. The frontend fallback (wraps non-JSON
as {message: raw_text}) provides additional safety.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Add widget E2E tests + strict tools + Agent Chat JSON mode
New Playwright tests (23 total, +3):
- 'renders bar-chart and pie-chart from x-ui annotations' — injects
mock agent with output_schema containing x-ui widgets, verifies
SVG rendering for pie/bar charts, stat-card with label, badges
- 'renders raw JSON for object data' — verifies auto-detection:
objects render as formatted JSON in pre blocks
- 'falls back gracefully for non-JSON output' — verifies plain
markdown string wraps as {message: text} and renders correctly
Agent Chat (agents.py) fixes:
- Added JSON output mode (response_format: json_object)
- Added strict=True tool binding for compatibility
- Matches the same pattern as agent_builder
Strict tool binding (react_runner.py):
- build_react_agent pre-binds tools with strict=True
- Required for OpenAI JSON mode (response_format: json_object)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix NameError: OpenAICallLoggingCallback was removed but still referenced
The class was deleted in the dead code cleanup but agents.py still
used it. Replaced with make_llm_logging_callback from agent_builder.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Add 'Show in Menu' — agents appear as tabs in navigation
When an agent has show_in_menu=true, it appears as a tab in the
main navigation bar. Clicking it opens a dedicated run page with
just the input field, run button, and SchemaRenderer output.
Backend:
- AgentDefinition: new show_in_menu bool field (default false)
- AgentDefinitionCreate/Update: show_in_menu parameter
- Migration for existing DBs
- Service wires it through create/update
Frontend:
- WorkbenchPage: 'Show in menu' checkbox in create form
- App.jsx: fetches agents with show_in_menu=true, injects as tabs
- AgentRunPage.jsx: simple standalone run page (title, description,
optional input, run button, SchemaRenderer output)
- Dynamic routes: /agent-run/{agentId}
E2E test:
- Creates agent via API with show_in_menu=true
- Verifies tab appears in navigation with agent name
- Clicks tab, verifies AgentRunPage renders
- Runs agent (mocked), verifies output with SchemaRenderer
24 Playwright + 133 backend = 157 tests pass.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Add missing tools to chat agent: csv_sla_breach_tickets, csv_ticket_stats
The SLA Breach page was slow because the chat agent (agents.py)
didn't have the csv_sla_breach_tickets tool. The prompt said
'call csv_sla_breach_tickets' but the tool didn't exist, so the
LLM tried to replicate SLA breach logic manually using
csv_list_tickets — fetching many tickets and reasoning over them.
Now the chat agent has all 6 CSV tools matching the operations:
- csv_list_tickets (existing)
- csv_get_ticket (existing)
- csv_search_tickets (existing)
- csv_ticket_fields (existing)
- csv_sla_breach_tickets (NEW — pre-computed, ~1000 tokens)
- csv_ticket_stats (NEW — aggregated stats, ~350 bytes)
Expected improvement: 1 tool call (~1000 tokens) instead of
multiple list calls + manual reasoning (~30-60K tokens).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Add ticket detail modal and enhance CSV ticket table functionality
Signed-off-by: Andre Bossard <anbossar@microsoft.com>
* Refactor CSVTicketTable component: reorder DialogActions import for consistency
Signed-off-by: Andre Bossard <anbossar@microsoft.com>
* Add reasoning_effort config + new tools for major speed improvement
Performance:
- reasoning_effort='low' as default — reduces gpt-5-nano from
512 reasoning tokens (~7s) to 0-192 tokens (~1-3s) per LLM call
- Configurable per agent: low (fast), medium, high (deep), default
- Both agent_builder and legacy chat agent use reasoning_effort='low'
New tools:
- csv_count_tickets: count matching tickets WITHOUT returning data.
Lets the LLM check 'how many VPN tickets?' (~50 tokens) before
deciding to fetch details (~5000 tokens)
- csv_search_tickets_with_details: search + return full details
(notes, resolution, description) in ONE call. Eliminates the
N × csv_get_ticket drill-down pattern that caused the
'Ticket Knowledgebase Creator' to make 5+ sequential LLM calls
Impact on 'Ticket Knowledgebase Creator' agent:
Before: search(compact) → get_ticket × N → generate = 5+ LLM calls × ~5s = 25s+
After: search_with_details(query, limit=10) → generate = 2 LLM calls × ~2s = 4s
Also fixed: removed stale response_format: json_object from build_llm
(was causing strict tool errors).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Update incident details in FALL_2_HARDWARE_PERIPHERIE and FALL_3_ZUGRIFF_BERECHTIGUNG documentation for consistency and clarity
Signed-off-by: Andre Bossard <anbossar@microsoft.com>
* Fix: all E2E tests now clean up created agents
Two tests were creating agents via the UI but not deleting them,
leaving orphans in the DB after each test run:
- 'runs an agent and appends output to run button'
- 'requires and forwards configured run input'
Added Delete button clicks at the end of both tests.
All 10 agent-creating tests now verified to clean up.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Rewrite workbench e2e tests for tabbed UI
- Add helpers: goToCreateTab, goToAgentsTab, createAgent, createAgentViaAPI, deleteAgentViaAPI, mockEmptyRuns
- Update 'creates and deletes' to use Create Agent tab and agent cards
- Update 'runs an agent' to verify output in RunsSidePanel
- Update 'requires input' to use card inline input field + Go button
- Update 'suggest schema' to navigate to Create tab first
- Update 'failure handling' to check error in run detail panel
- Refactor SchemaRenderer tests to use setupSchemaTest helper (API-created agents, run output in side panel)
- Keep Agent Chat UI and Show in Menu tests unchanged
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* feat: redesign workbench with agent cards, runs side panel, and tabbed layout
- Split WorkbenchPage into tabbed UI: Agents (cards grid) + Create Agent
- AgentCardsPanel: icon cards with Run/Edit/Delete buttons per agent
- RunsSidePanel: scrollable run history with click-to-view output
- AgentEditDialog: edit existing agents via dialog
- AgentCreateForm: extracted creation form (reusable for create + edit)
- Added API functions: updateWorkbenchAgent, listAllRuns, getRun
- All 47 Playwright tests pass (12 workbench tests updated for new UI)
- Removed Ollama references from setup.sh and package.json
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: LiteLLM fallback in agent_builder + add live lifecycle test
- Fixed agent_builder/engine/react_runner.py: ChatLiteLLM when no API key
- Fixed agent_builder/service.py: removed hard OpenAI key requirement
- Fixed agent_builder/chat_service.py: same
- Fixed RunsSidePanel output parsing for raw string output
- Added full lifecycle e2e test (live LLM): create → run → edit → re-run → verify history → delete
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* feat: suggest schema & tools, default no tools, pure function refactor
- 'Suggest Schema & Tools' button: LLM suggests output schema AND tool selection
- Backend: _build_suggest_prompt and _parse_suggest_response as pure functions
- Frontend: tools default to empty, populated by suggest response
- RunsSidePanel: pure calculations extracted (buildAgentMap, sortRunsNewestFirst,
resolveOutputSchema, resolveAgentName, parseRunOutput, formatRelativeTime)
- All 49 Playwright tests pass (2 live LLM tests included)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: result dialog, chart rendering, markdown fence parsing
- Run results now open in a large Dialog (900px wide, 85vh max)
- Fixed parseRunOutput: strips markdown code fences from LLM output
- Fixed PieChartWidget: filters non-numeric values, formats labels
- Fixed BarChartWidget: accepts object {key: number} in addition to arrays
- Chart containers: 300px height, 600px max-width
- Tests: close dialog before cleanup (dialog blocks pointer events)
- All 49 Playwright tests pass
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* feat: all-live Playwright tests, result dialog fix, runs panel fix
- Rewrote workbench tests: ZERO mocks, all 8 tests use live LLM
- Fixed RunsSidePanel: min-height for layout, runs visible on load
- Fixed parseRunOutput: strips markdown fences from LLM output
- Fixed chart widgets: pie/bar handle non-numeric values, proper sizing
- Fixed dialog close: tests use X button (in viewport) not Close (scrolled)
- Total: 43 tests, all passing, all live (1.1 min)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* refactor: extract shared parseRunOutput, add delete-all-runs
- Extracted parseRunOutput (fence-stripping + JSON parsing) into
outputUtils.js — shared by RunsSidePanel and AgentRunPage
- Fixed AgentRunPage (show_in_menu): renders markdown instead of raw JSON
- Added DELETE /api/workbench/runs endpoint + trash button in Runs panel
- Runs panel: min-height 500px so content is visible on load
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* feat: add SSE activity monitor, settings page, agent templates & run history
- Agent Activity page with real-time SSE event stream (tool calls, LLM
thinking, run lifecycle), filterable by run_id via URL query param
- EventBus pub/sub + StreamingCallbackHandler wired into ReAct engine
- Settings page: drag-and-drop tab reorder, hide/show toggles, icon
picker (57 FluentUI icons), persisted to localStorage
- Agent templates dropdown (KBA from tickets, worklog stats, next step
advisor) pre-fills the create agent form
- AgentRunPage now shows filtered run history with detail dialog and
link to Activity page filtered by run_id
- 19 new Playwright E2E tests (8 activity + 11 settings)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* feat: add Support Workflow canvas page with interactive editor
Purely browser-side workflow visualization using HTML Canvas:
- 5 node types: Start, End, Action, Decision, Wait (each with
distinct shape and color)
- Drag-and-drop to reposition nodes
- Shift+drag to create connections between nodes
- Double-click to rename nodes inline
- Animate button shows flowing dots along edges
- Toolbar to add/delete nodes, reset to default workflow
- Default workflow: Ticket Created → Auto-Classify → Priority
decision → L1/L2 paths → Resolved decision → Close/Reopen
- 9 Playwright E2E tests with screenshots
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* feat: metro-map workflow with presets, color picker, agent assignment
Rewrite WorkflowPage as metro-map style inspired by Incident &
Problem Solving methodology:
- 3 workflow presets: Incident Solving, Problem Solving, Change Mgmt
- Metro station circle nodes with thick colored edge lines
- Edge color inherited from outgoing node
- Click node → dialog with color picker (8 colors) and agent selector
(10 agent presets)
- Agent indicator dot on nodes with assigned agents
- Color legend auto-generated from used colors
- 12 Playwright E2E tests covering presets, node config, animation
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* feat: friendlier workflow editor — connect mode, double-click add, dialog edges
- Connect Mode toggle button: click source node then target to draw edge
(no shift key needed). Crosshair cursor + green '+' hint on target.
- Double-click empty canvas area to add a node at that position
- Node dialog now has 'Connect to…' section with buttons for each
unconnected node — draw edges without touching the canvas
- Add Node button opens config dialog immediately for the new node
- Dynamic help text updates based on current mode
- Escape key exits connect mode
- Updated Playwright tests for new UX
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* feat: add 'Improve my Prompt' button to Agent Fabric
LLM-powered prompt improvement following 2025 best practices:
- Backend: /api/workbench/improve-prompt endpoint + service method
that rewrites prompts with clear role, goals, numbered steps,
tool references, output format, and constraints
- Frontend: '✨ Improve my Prompt' button below the system prompt
textarea, disabled when empty, replaces prompt with improved version
- 4 Playwright E2E tests with before/after screenshots
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: prompt improvement skips output format (handled by schema)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: improve-prompt uses selected tools, not all available
Pass tool_names from frontend form state so the LLM only references
tools the user actually selected for this agent.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: remove maxHeight on tools list to avoid scrolling
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: replace worklog template with Topic & Product Analysis
Worklog columns in data.csv are all empty/zero. New template analyzes
topics, products, services, priority distribution, and group workload
using data that actually exists in the CSV.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Signed-off-by: Andre Bossard <anbossar@microsoft.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>1 parent 7ac9bb4 commit 1416147
78 files changed
Lines changed: 10664 additions & 2397 deletions
File tree
- backend
- agent_builder
- engine
- models
- persistence
- tests
- tools
- agent_workbench
- tests
- docs
- ticker_reminder
- frontend
- src
- features
- activity
- csvtickets
- settings
- workbench
- workflow
- services
- tests/e2e
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
This file was deleted.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
0 commit comments