Skip to content

(feature): add initial Runs dashboard#161

Merged
krisztianfekete merged 3 commits into
mainfrom
feature/evalruns-dashboard
Jul 2, 2026
Merged

(feature): add initial Runs dashboard#161
krisztianfekete merged 3 commits into
mainfrom
feature/evalruns-dashboard

Conversation

@krisztianfekete

@krisztianfekete krisztianfekete commented Jun 30, 2026

Copy link
Copy Markdown
Contributor

This PR adds a dashboard to be able to explore historical evaluations backed by PG.

image

Fixes #140

@krisztianfekete krisztianfekete force-pushed the feature/evalruns-dashboard branch from ac7c667 to e4307f2 Compare July 1, 2026 09:55
@krisztianfekete krisztianfekete marked this pull request as ready for review July 2, 2026 14:53
@krisztianfekete krisztianfekete requested a review from Copilot July 2, 2026 14:54

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds an initial “Run history” dashboard to the UI, backed by the durable Postgres run storage, and extends the backend + streaming pipeline so runs can be grouped and trended over time (including stable agent identity via service.name).

Changes:

  • Add a new Runs dashboard view (list + charts + per-run detail) powered by GET /api/runs and GET /api/runs/{id}/results.
  • Track and propagate a “golden/reference trace” so it’s plotted in performance charts but excluded from pass/fail scoring.
  • Persist richer run summaries (per-metric aggregates + agent identity) and ensure service.name survives streaming trace serialization.

Reviewed changes

Copilot reviewed 30 out of 30 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
ui/src/lib/types.ts Adds persisted run-history types (Run, RunSummary, RunResultRow, etc.) and wires runs into ViewType.
ui/src/context/TraceProvider.tsx Adds goldenTraceId to state and clears it on clearResults; exposes setGoldenTraceId.
ui/src/context/TraceContext.tsx Extends TraceState/actions with goldenTraceId + setter.
ui/src/config.ts Adds /api/runs endpoint URL.
ui/src/components/streaming/LiveStreamingView.tsx Includes golden trace in uploaded traces and records goldenTraceId for later scoring/chart behavior.
ui/src/components/sidebar/Sidebar.tsx Adds “Run History” navigation item and sidebar section mapping for the runs view.
ui/src/components/runs/RunsView.tsx New Runs dashboard page: load runs, group/filter, show summary stats + trend charts + table + drill-in.
ui/src/components/runs/RunsHistoryTable.tsx New table rendering run rows with status, group keys, pass rate bar, duration, and models.
ui/src/components/runs/runHistory.ts New helpers for grouping, pass rate/duration formatting, frequency-based legends, and status/chart colors.
ui/src/components/runs/RunDetailView.tsx New per-run detail view showing configuration, per-case results, and persisted tool-trajectory diffs.
ui/src/components/runs/PerMetricTrendChart.tsx New chart.js per-metric avg score trends across runs.
ui/src/components/runs/PassRateTrendChart.tsx New chart.js pass-rate trends (per agent line where available).
ui/src/components/dashboard/TraceTable.tsx Adds trajectory-diff badge and expands rows to show tool-trajectory mismatch details.
ui/src/components/dashboard/PerformanceCharts.tsx Labels/sorts the golden trace as “(reference)” and prioritizes it in chart ordering.
ui/src/components/dashboard/DashboardView.tsx Excludes golden trace from scoring/table rows while still passing it to performance charts for plotting.
ui/src/App.tsx Routes the new runs view to RunsView.
ui/src/api/client.ts Adds listRuns, getRun, getRunResults plus a StorageUnavailableError for 503 durable-storage gating.
src/agentevals/trace_metrics.py Extracts stable agent identity using service.name (with fallback) and adds it to extracted metadata.
src/agentevals/trace_attrs.py Introduces OTEL_SERVICE_NAME = "service.name" constant.
src/agentevals/streaming/ws_server.py Re-attaches service.name resource attribute onto each span when serializing JSONL traces.
src/agentevals/runner.py Adds service_name/agent_name onto TraceResult using extract_agent_identity.
src/agentevals/run/result_builder.py Ensures summary/result status mapping consistency; adds per-metric aggregates and agent list to persisted run summary.
src/agentevals/api/streaming_routes.py Persists “evaluate sessions” streaming runs when durable storage is configured; reuses canonical trace serializer for /get-trace.
examples/zero-code-examples/strands/run.py Sets OTEL_SERVICE_NAME by default for stable run grouping.
examples/zero-code-examples/pydantic-ai/run.py Sets OTEL_SERVICE_NAME by default for stable run grouping.
examples/zero-code-examples/openai-agents/run.py Sets OTEL_SERVICE_NAME by default for stable run grouping.
examples/zero-code-examples/ollama/run.py Sets OTEL_SERVICE_NAME by default for stable run grouping.
examples/zero-code-examples/langchain/run.py Sets OTEL_SERVICE_NAME by default for stable run grouping.
examples/zero-code-examples/adk/run.py Sets OTEL_SERVICE_NAME by default for stable run grouping.
examples/dice_agent/agent.py Updates the example agent model identifier.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread ui/src/components/dashboard/TraceTable.tsx
@krisztianfekete krisztianfekete merged commit 5b75cf1 into main Jul 2, 2026
6 checks passed
@krisztianfekete krisztianfekete deleted the feature/evalruns-dashboard branch July 2, 2026 15:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add historical view to data backed by PostgreSQL

2 participants