Commit 39164e2
committed
feat(gastown): measure true container cold-start and mayor-ready latency
The prior container-startup panels measured DO→container RPC round-trips
(/health and /agents/start), not actual cold-start time — /health is
truncated at a 5s client timeout so p99 was bounded below the true
cold-start budget, and the dashboard queries filtered on blob8/blob9
values that don't exist in the AE schema so the panels showed nothing.
This replaces those with two metrics that answer the original questions:
1. container.cold_start — TownContainerDO.warmUp() invokes the
Container class's startAndWaitForPorts() directly and times it.
Emitted only when the container was actually started (state !=
healthy), so the quantiles reflect real cold starts without being
capped by an arbitrary client-side timeout.
2. mayor.session_ready — container stamps mayorReadyAt when the first
mayor agent transitions to 'running' and exposes it via /health.
Town DO reads it and emits durationMs = mayorReadyAt - startedAt
exactly once per container lifetime (deduped in DO storage keyed
by containerStartedAt).
Dashboard fixes:
- Rename 'Container Startup Latency' row to 'DO → Container RPC
Latency' and clarify panel titles so operators don't read p99 off
those and think it's cold-start time.
- Fix broken success/failure filters: blob8='ok' / blob9='true' →
blob5='' (error absent), blob5!='' (error present).
- Convert quantile queries from label-column style (which collapsed
all three series to 'latency_ms') to column-name style (AS p50/p90/
p99), so the legend actually distinguishes the percentiles.
- Add new 'Container Cold Start & Mayor Ready' row with p50/p90/p99
panels for the two new events.1 parent 5f6f613 commit 39164e2
6 files changed
Lines changed: 448 additions & 32 deletions
File tree
- services/gastown
- container/src
- src/dos
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
10 | 10 | | |
11 | 11 | | |
12 | 12 | | |
| 13 | + | |
13 | 14 | | |
14 | 15 | | |
15 | 16 | | |
| |||
221 | 222 | | |
222 | 223 | | |
223 | 224 | | |
| 225 | + | |
224 | 226 | | |
225 | 227 | | |
226 | 228 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
78 | 78 | | |
79 | 79 | | |
80 | 80 | | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
81 | 101 | | |
82 | 102 | | |
83 | 103 | | |
| |||
1044 | 1064 | | |
1045 | 1065 | | |
1046 | 1066 | | |
| 1067 | + | |
| 1068 | + | |
| 1069 | + | |
1047 | 1070 | | |
1048 | 1071 | | |
1049 | 1072 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
165 | 165 | | |
166 | 166 | | |
167 | 167 | | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
168 | 172 | | |
169 | 173 | | |
170 | 174 | | |
| |||
0 commit comments