Show AgentX cumulative convergence whiskers / 展示 AgentX 累计收敛须线

[cquil11]

Summary

• Flatten and allowlist the runner's cumulative-convergence diagnostics through ETL, API rows, benchmark transforms, and chart-ready types.
• Resolve the convergence result for the active P75/P90 TTFT, end-to-end latency, or interactivity x-axis metric.
• Draw thin horizontal whiskers only for stabilized points on the existing Pareto frontier. Whisker endpoints expand the x-domain but never participate in frontier selection.
• Render no whisker when a metric was evaluated but did not stabilize; the tooltip states that outcome explicitly.
• Preserve the older non-overlapping-window diagnostics in storage for drift analysis, but stop using their total range as the Pareto whisker.
• Support both official data and ?unofficialrun= overlays, including overlay colors, hardware visibility, per-run dismissal, zoom, and tooltip behavior.
• Add English and Simplified Chinese legend and tooltip explanations.

Convergence construction

For each P75/P90 latency metric:

1. Recompute the metric cumulatively every 300 seconds.
2. Use the final complete checkpoint as the reference.
3. Find the earliest checkpoint with at least 1,200 seconds of later confirmation where every later estimate remains within a symmetric multiplicative ±5% band of the final estimate.
4. If it stabilizes, draw the minimum-to-maximum cumulative estimate after that checkpoint. Otherwise, draw no whisker.

The multiplicative band is checked with absolute log ratios so latency and reciprocal interactivity have identical stabilization timing.

Statistical caveat

This is a retrospective within-run convergence diagnostic, not a confidence interval or a prediction of rerun-to-rerun variability. It answers “when did this run's cumulative estimate settle, and how much did it move afterward?” It does not estimate population uncertainty; that still requires independent randomized repetitions and run-level/root-cluster inference.

Isolated Neon backfill

• Project: silent-pond-29172997
• Branch: agentx-uncertainty-db-mvp
• Branch ID: br-wandering-pine-aih0gnca
• Endpoint: ep-still-boat-ai7xcrdl
• 339 AgentX rows evaluated.
• Stabilized P90 counts: TTFT 75, end-to-end latency 106, interactivity 177.
• latest_benchmarks was refreshed on this branch.
• A branch-guarded rerun for point 436361 computed one patch and updated zero rows, confirming idempotence.
• Production branch br-restless-voice-ai2vkru2 still contains zero convergence fields.

Point 436361: stabilized at 1,200 seconds with 407 valid samples; post-stabilization P90 interactivity span 43.36036–46.97824 tok/s/user; maximum later deviation 0.04385.

Preview verification

Works for both official runs and ?unofficialrun= overlays — verified at:

• https://inferencemax-app-git-codex-agentx-uncerta-689510-semianalysisai.vercel.app/inference
• https://inferencemax-app-git-codex-agentx-uncerta-689510-semianalysisai.vercel.app/inference/agentic/436361
• https://inferencemax-app-git-codex-agentx-uncerta-689510-semianalysisai.vercel.app/zh/inference

The deployed official chart renders 35 convergence whiskers. Point 436361 shows the exact construction above in English and Chinese, with no Next.js error overlay. A real historical overlay (?unofficialrun=26312107787) loads successfully; as expected, it has no convergence whisker because that artifact predates these runner fields. Convergence-bearing overlay whiskers, colors, zoom, and dismissal are verified by the component and E2E fixture paths.

Validation

• App unit tests: 2,743 passed
• DB unit tests: 372 passed
• Constants unit tests: 33 passed
• MCP unit tests: 25 passed
• Cypress component tests: 182/182 passed
• Targeted AgentX E2E: 16/16 passed
• Lint, formatting, typecheck, and production build against the Neon feature branch: passed
• Full Cypress baseline: 496/501; the only five failures are in the untouched evaluation-chart spec, whose own benchmark selection removes #evaluation-chart and persists that no-data state. The changed AgentX E2E spec passed all 16 tests in that run.

中文说明

• 将 runner 生成的累计收敛诊断依次展平并加入 ETL、API 行、基准测试转换和图表类型。
• 根据当前 X 轴选择，解析对应的 P75/P90 TTFT、端到端延迟或交互性收敛结果。
• 仅对已达到稳定且位于现有 Pareto 前沿上的点绘制细水平须线。须线端点会扩展 X 轴范围，但不会参与 Pareto 前沿计算。
• 如果某项指标已完成评估但未达到稳定，则不绘制须线，并在 tooltip 中明确说明。
• 保留旧的互不重叠时间窗口诊断以供漂移分析，但不再把其完整范围作为 Pareto 须线。
• 同时支持官方数据和 ?unofficialrun= 非官方运行叠加层，包括叠加层配色、硬件可见性、单次运行关闭、缩放和 tooltip。
• 提供英文与简体中文的图例和 tooltip 说明。

收敛计算方法

针对每个 P75/P90 延迟指标：

1. 每 300 秒基于截至当时的全部有效请求重新计算累计指标。
2. 以最后一个完整检查点作为参考值。
3. 找到最早的检查点：其后至少还有 1,200 秒确认期，且所有后续累计估计值都位于最终值的对称乘法 ±5% 范围内。
4. 如果达到稳定，则绘制该检查点之后累计估计值的最小值到最大值；否则不绘制须线。

乘法范围通过绝对对数比判断，因此延迟及其倒数形式的交互性指标会在同一时间达到稳定。

统计说明

这是对单次运行内部收敛情况的回溯性诊断，并非置信区间，也不预测不同复跑之间的波动。它回答的是“本次运行的累计估计值何时趋于稳定，之后还移动了多少”。如需估计总体不确定性，仍需独立随机重复实验，并在运行层级或根轨迹聚类层级进行统计推断。

独立 Neon 回填分支

• 项目：silent-pond-29172997
• 分支：agentx-uncertainty-db-mvp
• 分支 ID：br-wandering-pine-aih0gnca
• Endpoint：ep-still-boat-ai7xcrdl
• 共评估 339 条 AgentX 记录。
• P90 达到稳定的数量：TTFT 75、端到端延迟 106、交互性 177。
• 已在该分支刷新 latest_benchmarks。
• 对点 436361 再次执行带分支保护的回填时，计算出 1 条 patch、更新 0 行，确认回填具有幂等性。
• 生产分支 br-restless-voice-ai2vkru2 仍然包含 0 个收敛字段。

点 436361：在 1,200 秒时达到稳定，使用 407 个有效样本；稳定后的 P90 交互性范围为 43.36036–46.97824 tok/s/user；后续最大偏差为 0.04385。

Preview 验证

官方运行和 ?unofficialrun= 叠加层均受支持，已在以下地址验证：

• https://inferencemax-app-git-codex-agentx-uncerta-689510-semianalysisai.vercel.app/inference
• https://inferencemax-app-git-codex-agentx-uncerta-689510-semianalysisai.vercel.app/inference/agentic/436361
• https://inferencemax-app-git-codex-agentx-uncerta-689510-semianalysisai.vercel.app/zh/inference

部署后的官方图表共渲染 35 条收敛须线。点 436361 的中英文 tooltip 均显示上述精确计算过程，页面没有 Next.js 错误浮层。真实历史叠加运行（?unofficialrun=26312107787）可以正常加载；由于该历史产物早于本次 runner 字段，因此按预期不显示收敛须线。带收敛字段的叠加须线、配色、缩放和关闭行为已通过组件测试和 E2E fixture 路径验证。

验证结果

• App 单元测试：2,743 项通过
• DB 单元测试：372 项通过
• Constants 单元测试：33 项通过
• MCP 单元测试：25 项通过
• Cypress 组件测试：182/182 通过
• AgentX 定向 E2E：16/16 通过
• Lint、格式检查、类型检查，以及连接 Neon 功能分支的生产构建：通过
• 完整 Cypress 基线：496/501；仅有 5 项失败来自未修改的 evaluation-chart spec，其自身选择了无数据的 benchmark，导致 #evaluation-chart 被移除并把该状态带入后续测试。本次修改的 AgentX E2E spec 在该轮完整测试中 16 项全部通过。

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
inferencemax-app	Ready	Preview, Comment	Jul 21, 2026 10:22pm