Skip to content

Commit 734e4f2

Browse files
committed
docs/tutorials: 4 new in-depth tutorials
Adds: - 07 dashboard_deployment: stand up vstack-dashboard in prod - 08 baselines_and_drift: per-pattern + fleet baseline + CI gates - 09 async_fanout: production-volume async patterns with rate limiting, bounded concurrency, back-pressure - 10 observability: structured logging, Prometheus metrics, OpenTelemetry tracing, cost dashboards, alerting
1 parent 49a1063 commit 734e4f2

4 files changed

Lines changed: 1199 additions & 0 deletions

File tree

Lines changed: 322 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,322 @@
1+
# Tutorial 7 — Deploying the vstack Dashboard
2+
3+
> Goal: stand up the vstack HTML dashboard in production. Covers
4+
> rendering, serving, embedding, and integrating with the diagnose
5+
> pipeline.
6+
7+
---
8+
9+
## What you'll build
10+
11+
By the end of this tutorial you'll have:
12+
13+
1. A static HTML report you can email or save.
14+
2. A long-running FastAPI dashboard server you can browse.
15+
3. A wired-up pipeline that captures every production run + renders
16+
a dashboard automatically.
17+
18+
---
19+
20+
## Part 1 — Static HTML rendering
21+
22+
The simplest dashboard surface: render a `DiagnoseReport` to a
23+
single self-contained HTML file.
24+
25+
### Setup
26+
27+
```bash
28+
pip install valanistack
29+
```
30+
31+
The dashboard module is bundled in vstack — no extras needed.
32+
33+
### Render
34+
35+
```python
36+
from vstack import diagnose
37+
from vstack.aar import AgentTrace, TraceStep
38+
from vstack.aar.clients import StubClient
39+
from vstack.dashboard import render_report
40+
41+
trace = AgentTrace(
42+
goal="...",
43+
steps=[
44+
TraceStep(type="thought", content="..."),
45+
TraceStep(type="tool_call", content="..."),
46+
TraceStep(type="observation", content="..."),
47+
],
48+
outcome="...",
49+
success=False,
50+
)
51+
52+
report = diagnose(trace=trace, llm_client=StubClient())
53+
html = render_report(report)
54+
55+
with open("report.html", "w") as f:
56+
f.write(html)
57+
```
58+
59+
Open `report.html` in any browser. The HTML is self-contained —
60+
all CSS + JS is inlined via Tailwind CDN + Chart.js CDN — and
61+
renders identically across browsers.
62+
63+
### Configure
64+
65+
```python
66+
from vstack.dashboard import render_report, DashboardConfig
67+
68+
config = DashboardConfig(
69+
title="Production Run 2026-06-09",
70+
badge_style="severity-tinted",
71+
chart_height=320,
72+
)
73+
html = render_report(report, config=config)
74+
```
75+
76+
### Multi-report overview
77+
78+
If you have many reports, render an overview page that links to each:
79+
80+
```python
81+
from vstack.dashboard import render_reports_overview
82+
83+
reports = [report1, report2, report3]
84+
overview_html = render_reports_overview(
85+
reports=reports,
86+
title="Production Runs This Week",
87+
)
88+
89+
with open("index.html", "w") as f:
90+
f.write(overview_html)
91+
```
92+
93+
The overview page shows a sparkline + severity counts per report.
94+
95+
---
96+
97+
## Part 2 — FastAPI dashboard server
98+
99+
For interactive browsing, run the dashboard as a long-running
100+
FastAPI server. Reports persist in-memory (or in a backing store)
101+
and are browsable via routes.
102+
103+
### Launch
104+
105+
```bash
106+
vstack-dashboard serve --port 7878
107+
# or:
108+
python -m vstack.dashboard serve --port 7878
109+
```
110+
111+
Browse `http://localhost:7878`.
112+
113+
### Routes
114+
115+
| Route | Purpose |
116+
|-----------------------------|--------------------------------------------------|
117+
| `GET /` | Overview of all stored reports |
118+
| `GET /runs` | List runs (paginated) |
119+
| `GET /runs/{report_id}` | Single report's full HTML render |
120+
| `GET /patterns` | Per-pattern explorer |
121+
| `GET /recipes` | Recipe catalog browser |
122+
| `POST /v1/reports` | Submit a new report |
123+
| `GET /v1/reports` | List report IDs (JSON) |
124+
| `GET /healthz` | Liveness check |
125+
126+
### Submit a report
127+
128+
```python
129+
import requests
130+
from vstack import diagnose
131+
132+
report = diagnose(trace=..., llm_client=...)
133+
134+
response = requests.post(
135+
"http://localhost:7878/v1/reports",
136+
json={"report_id": "run-001", "report": report.model_dump()},
137+
)
138+
print(response.json())
139+
# {"status": "stored", "url": "/runs/run-001"}
140+
```
141+
142+
### Persistence
143+
144+
By default the dashboard uses an in-memory LRU-ish store with
145+
capacity 1000. For persistence, configure a backing store via env:
146+
147+
```bash
148+
VSTACK_DASHBOARD_STORE=filesystem
149+
VSTACK_DASHBOARD_STORE_PATH=/var/lib/vstack-dashboard
150+
151+
vstack-dashboard serve --port 7878
152+
```
153+
154+
Reports persist as JSON under the configured path. The server
155+
loads them at startup.
156+
157+
---
158+
159+
## Part 3 — Wired-up production pipeline
160+
161+
For a full production pipeline that captures every run and
162+
renders a dashboard automatically:
163+
164+
```python
165+
from vstack import diagnose
166+
from vstack.aar import AgentTrace
167+
from vstack.dashboard import render_report
168+
import requests
169+
170+
DASHBOARD_URL = "http://dashboard.internal:7878"
171+
172+
173+
def diagnosed_production_run(agent_call_fn, **kwargs):
174+
"""Wraps any agent function with vstack diagnostics + dashboard."""
175+
trace_id = generate_id()
176+
177+
# Run the production agent.
178+
result, trace = agent_call_fn(**kwargs)
179+
180+
# Diagnose.
181+
report = diagnose(trace=trace, llm_client=llm)
182+
183+
# Submit to dashboard.
184+
submit_to_dashboard(trace_id, report)
185+
186+
return result
187+
188+
189+
def submit_to_dashboard(report_id: str, report) -> None:
190+
"""POST the report to the dashboard server."""
191+
try:
192+
response = requests.post(
193+
f"{DASHBOARD_URL}/v1/reports",
194+
json={"report_id": report_id, "report": report.model_dump()},
195+
timeout=5,
196+
)
197+
response.raise_for_status()
198+
except requests.RequestException as exc:
199+
# Dashboard submission is best-effort; don't fail production.
200+
logger.warning("Dashboard submission failed: %s", exc)
201+
```
202+
203+
### Severity-triggered alerts
204+
205+
```python
206+
def submit_to_dashboard_with_alerts(report_id: str, report) -> None:
207+
"""Submit + alert on high-severity findings."""
208+
submit_to_dashboard(report_id, report)
209+
210+
high = [f for f in report.findings if f.severity == "high"]
211+
if high:
212+
send_alert(
213+
channel="#agent-alerts",
214+
text=f"High-severity findings in run {report_id}: {len(high)}",
215+
dashboard_url=f"{DASHBOARD_URL}/runs/{report_id}",
216+
)
217+
```
218+
219+
---
220+
221+
## Embedding the dashboard
222+
223+
### In a Jupyter notebook
224+
225+
```python
226+
from IPython.display import HTML
227+
from vstack.dashboard import render_report
228+
229+
html = render_report(report)
230+
HTML(html)
231+
```
232+
233+
### In Streamlit
234+
235+
```python
236+
import streamlit as st
237+
from vstack.dashboard import render_report
238+
239+
html = render_report(report)
240+
st.components.v1.html(html, height=900, scrolling=True)
241+
```
242+
243+
### In a Slack message
244+
245+
```python
246+
# Render as image (requires headless Chrome / Playwright).
247+
from playwright.sync_api import sync_playwright
248+
from vstack.dashboard import render_report
249+
250+
html = render_report(report)
251+
with open("/tmp/report.html", "w") as f:
252+
f.write(html)
253+
254+
with sync_playwright() as p:
255+
browser = p.chromium.launch()
256+
page = browser.new_page()
257+
page.goto("file:///tmp/report.html")
258+
page.screenshot(path="/tmp/report.png", full_page=True)
259+
browser.close()
260+
261+
# Then send the PNG via Slack API.
262+
```
263+
264+
---
265+
266+
## Production deployment checklist
267+
268+
- [ ] Dashboard server behind reverse proxy with TLS.
269+
- [ ] Persistence backing store configured (`VSTACK_DASHBOARD_STORE`).
270+
- [ ] Backup policy for the backing store.
271+
- [ ] Auth in front of POST endpoints (the in-process auth is for
272+
sketch only; use your gateway).
273+
- [ ] Rate limit on POST endpoints if exposed to user-facing
274+
agents.
275+
- [ ] Monitoring for the `/healthz` endpoint.
276+
- [ ] Log aggregation for the JSON access logs.
277+
- [ ] Retention policy for old reports (filesystem store grows
278+
unbounded by default).
279+
280+
---
281+
282+
## Troubleshooting
283+
284+
### Dashboard renders blank
285+
286+
The dashboard uses Tailwind CDN + Chart.js CDN. If the rendered
287+
HTML loads in a browser with no network access, charts won't render.
288+
For air-gapped deployments, use the `bundle_assets=True` mode:
289+
290+
```python
291+
html = render_report(report, config=DashboardConfig(bundle_assets=True))
292+
```
293+
294+
This inlines Tailwind and Chart.js into the HTML output. The
295+
file is bigger (~250KB instead of 20KB) but self-contained.
296+
297+
### Charts wrong colour
298+
299+
The dashboard auto-detects the user's prefers-color-scheme. Force
300+
a theme with:
301+
302+
```python
303+
config = DashboardConfig(theme="dark") # or "light"
304+
html = render_report(report, config=config)
305+
```
306+
307+
### Custom CSS
308+
309+
```python
310+
config = DashboardConfig(custom_css="""
311+
body { font-family: 'Inter', sans-serif; }
312+
.severity-high { background: #b91c1c; }
313+
""")
314+
```
315+
316+
---
317+
318+
## See also
319+
320+
- Tutorial 6: FastAPI deployment
321+
- Surface reference: `docs/surfaces/dashboard.md`
322+
- Source: `_dashboard/lib/`

0 commit comments

Comments
 (0)