feat(overview): create overview page / 创建总览页面#611
Conversation
|
Deployment failed with the following error: View Documentation: https://vercel.com/docs/accounts/team-members-and-roles |
|
Preview status update:
The protected Vercel Preview is still required for final server-loader and responsive validation. |
edwingao28
changed the title
5160d07 to
a854318
Compare
edwingao28
changed the title
a854318 to
785c1af
Compare
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
785c1af to
4d85aba
Compare
…and result-level evidence
4d85aba to
eec67f3
Compare
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: eec67f33dd
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| const secondaryMeasured = new Set(model.secondary?.measuredHardware); | ||
| const entries = model.notRanked | ||
| .filter((entry) => !secondaryMeasured.has(entry.hardwareLabel)) |
There was a problem hiding this comment.
When the secondary precision is in ranked mode, measuredHardware still includes every hardware with any row at that precision, but the secondary section only renders exact ranked chips. For a hardware that has no primary exact @50 result and only has secondary rows that are standard-decode or otherwise lack an exact @50 read, this filter removes its model.notRanked entry even though no secondary chip or coverage line names it, so that hardware silently disappears instead of showing its reason.
Useful? React with 👍 / 👎.
| for (let i = 0; i < last; i++) { | ||
| if (frontier[i].interactivity <= tier) lo = i; | ||
| } | ||
| return [frontier[lo], frontier[lo + 1]]; |
There was a problem hiding this comment.
For tiers that exactly match an interior frontier knot, the loop still returns that knot plus the next one, so the overview/TCO JSON reports a date range even though the displayed value is an exact measured point. This shows misleading evidence dates whenever, for example, a config has knots at 30/50/75/100 and the 50 tok/s/user row's date differs from the 75 row; the bracket should collapse to the matching knot before falling back to an interpolation pair.
Useful? React with 👍 / 👎.
|
Claude finished @edwingao28's task in 5m 20s —— View job Review — PR #611 Executive Overview pageRuntime verified on I confirmed the two open bot findings (Cursor Bugbot + Codex agree on both) and posted precise fixes inline:
Both are 🟡 (correctness/honesty of displayed evidence, not crashes). No other blocking issues found — no missing-test gaps (new lib code has colocated tests, the UI has an e2e), no resource/hook/security concerns in the new code.
|
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
| let lo = 0; | ||
| for (let i = 0; i < last; i++) { | ||
| if (frontier[i].interactivity <= tier) lo = i; | ||
| } | ||
| return [frontier[lo], frontier[lo + 1]]; |
There was a problem hiding this comment.
🟡 WARNING: Exact interior-knot reads report a cross-day evidence range they didn't interpolate. (Confirms the Cursor Bugbot / Codex finding.)
Why it matters: When a tier lands exactly on an interior frontier knot, hermiteInterpolate returns that knot's measured throughput exactly — it's a single measured point, no interpolation. But bracketKnots only collapses to one knot at the two endpoints; for an interior exact hit it still returns [frontier[lo], frontier[lo+1]], so evidenceRange widens to {from, to} spanning two dates. The overview then renders a date range for a value the PR body promises is a same-day exact read ("Same-day reads show one date, cross-day interpolations show a range"). Low trigger probability for the live overview (interactivity is a float rarely exactly 30/50/75/100), but this is the shared helper backing the public tco-feed API too.
Fix:
| let lo = 0; | |
| for (let i = 0; i < last; i++) { | |
| if (frontier[i].interactivity <= tier) lo = i; | |
| } | |
| return [frontier[lo], frontier[lo + 1]]; | |
| let lo = 0; | |
| for (let i = 0; i < last; i++) { | |
| if (frontier[i].interactivity <= tier) lo = i; | |
| } | |
| // An exact hit on an interior knot is a measured point, not an interpolation — | |
| // collapse to that single knot so its evidence date isn't widened to a range. | |
| if (frontier[lo].interactivity === tier) return [frontier[lo], frontier[lo]]; | |
| return [frontier[lo], frontier[lo + 1]]; |
| // Hardware already named by the secondary section — its ranked rows or its | ||
| // coverage 'measured' list — is not repeated here; dedup against the secondary's | ||
| // measured labels so each hardware surfaces in exactly one place. | ||
| const secondaryMeasured = new Set(model.secondary?.measuredHardware); |
There was a problem hiding this comment.
🟡 WARNING: A ranked secondary silently drops secondary-measured hardware from the not-ranked line, breaking the page's "every hardware accounted for" guarantee. (Confirms the Cursor Bugbot / Codex finding.)
Why it matters: secondaryMeasured is built from model.secondary.measuredHardware, which lists every hardware with any row at the secondary precision. When secondary.state === 'coverage' this is correct — the coverage line names all of them. But when secondary.state === 'ranked', SecondarySection renders only the exact-@50 ranked chips (overview-scorecard.tsx:417-426), not the full measured list. A hardware that (a) has no exact @50 read at the primary precision and (b) is measured at the secondary precision but has no exact @50 secondary read (e.g. standard-decode-only or clamped there) is:
- not an exact primary chip,
- not a ranked secondary chip,
- filtered out of
NotRankedLinehere because its label is inmeasuredHardware, - and gets no coverage line (that only renders in
coveragestate).
→ it disappears entirely. buildNotRanked (overview-data.ts:846-867) already excludes only the exact-@50 secondary hardware, so in ranked state this extra measuredHardware dedup is over-aggressive.
Not reproducible on today's real data (every model resolves to a single ranked precision — all secondaries are coverage), but the ranked branch is live code and this contradicts a headline invariant of the feature.
Fix: only dedup against the full measured list in coverage state; buildNotRanked already handles the ranked case.
| const secondaryMeasured = new Set(model.secondary?.measuredHardware); | |
| // Only the coverage-state secondary names its full measured list. A ranked | |
| // secondary shows just exact-@50 chips, and buildNotRanked has already | |
| // excluded those — so deduping against measuredHardware there would wrongly | |
| // drop secondary-measured hardware that has no exact @50 read at all. | |
| const secondaryMeasured = new Set( | |
| model.secondary?.state === 'coverage' ? model.secondary.measuredHardware : [], | |
| ); |
Worth a unit test in overview-data.test.ts pinning a ranked secondary where a third hardware measured at the secondary precision (no exact @50) still surfaces in notRanked.
|
Claude finished @edwingao28's task in 2m 13s —— View job Re-review — PR #611 Executive Overview pageRe-reviewing the one new commit since my last review ( What changed: Runtime verified on However — neither open finding from the last round was addressed:
No new issues introduced by the ∞-chip commit. Both remaining items are 🟡 (honesty of displayed evidence, not crashes) — recommend fixing both before merge and adding the
|
| // Hardware already named by the secondary section — its ranked rows or its | ||
| // coverage 'measured' list — is not repeated here; dedup against the secondary's | ||
| // measured labels so each hardware surfaces in exactly one place. | ||
| const secondaryMeasured = new Set(model.secondary?.measuredHardware); |
There was a problem hiding this comment.
🟡 WARNING (still open after 69ae73c): the ∞-chip rewrite kept the over-aggressive dedup, so a ranked secondary still drops secondary-measured hardware from the not-ranked line.
Why it matters: measuredHardware lists every hardware with a row at the secondary precision, but a ranked secondary renders only exact-@50 chips (overview-scorecard.tsx:417-426). A hardware that has no exact @50 read at the primary and no exact @50 read at the secondary (e.g. standard-decode-only there) is not a primary chip, not a ranked secondary chip, and now filtered out of this ∞ line too — so it gets no chip and no reason anywhere, breaking the "every hardware accounted for" guarantee. buildNotRanked (overview-data.ts:846-857) already excludes only the exact-@50 ranked secondary hardware, so this extra dedup is only correct in coverage state. (Not reproducible on today's real data — all secondaries resolve to coverage — but the ranked branch is live code.)
Fix: only dedup against the full measured list in coverage state.
| const secondaryMeasured = new Set(model.secondary?.measuredHardware); | |
| // Only a coverage-state secondary names its full measured list; a ranked | |
| // secondary shows just exact-@50 chips, which buildNotRanked already excludes. | |
| // Deduping against the full measured list in ranked state would drop | |
| // secondary-measured hardware that has no exact @50 read from every surface. | |
| const secondaryMeasured = new Set( | |
| model.secondary?.state === 'coverage' ? model.secondary.measuredHardware : [], | |
| ); |
Worth a unit test in overview-data.test.ts pinning a ranked secondary where a third hardware measured at the secondary precision (no exact @50) still surfaces in notRanked.
|
Claude finished @edwingao28's task in 2m 11s —— View job Re-review — PR #611 Executive Overview pageRe-reviewing the new commit Both previously-flagged findings are resolved:
Runtime verified on
CI: green — Lint, Typecheck+Unit, Component, E2E chrome (all shards) + firefox all passing (a couple of jobs still finishing, none failing). No new blocking or warning issues found in the rewrite. The data-layer invariants (exact-vs-clamped classification via
|
![github-code-quality[bot]](https://avatars.githubusercontent.com/u/9919?s=60&v=4){kind=small}
|
Claude finished @edwingao28's task in 56s —— View job Re-review — PR #611 Executive Overview pageRe-reviewing the one new commit Both findings resolved, and both changes verified safe:
CI: green across the board — Lint, Typecheck+Unit, Component, CodeQL, E2E chrome (4 shards) + firefox (4 shards), Vercel Preview all passing. (Claude review / Cursor Bugbot still pending, as expected.) No new issues introduced. The two data-honesty findings from earlier rounds (
|
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 2 potential issues.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 7378a3b. Configure here.
| }; | ||
| }), | ||
| latestDate: feed[0].latest_date, | ||
| oldestFrontierDate: feed[0].oldest_frontier_date, |
There was a problem hiding this comment.
Unused config fields computed
Low Severity
buildConfigResult populates several OverviewConfigResult fields that nothing in the overview UI or link builders reads, including offloadMode, isMultinode, GPU counts, totalGpu, parallelism, image, and oldestFrontierDate. That adds maintenance surface and wasted work on every page assemble.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit 7378a3b. Configure here.
| ))} | ||
| <span className="flex items-center gap-2 px-3 sm:hidden"> | ||
| <MinecraftToggles /> | ||
| </span> |
There was a problem hiding this comment.
Menu audio controls undersized
Low Severity
This PR moves MinecraftToggles into the mobile menu for the 320px layout and raises other header controls to a 44px touch target, but the audio buttons still use compact p-2 sizing. In minecraft mode on small screens those menu controls remain below the WCAG minimum the new tests encode.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit 7378a3b. Configure here.
1 participant
Adds
/overview(and/zh/overview): a server-rendered scorecard that answers one question per model — if you deploy it today, how much capacity does each platform deliver?What it shows
FRAMEWORK · PRECISION, with the full stack on the link title. This measures deployable capacity, not isolated silicon, and the methodology note on the page says so.?tier=30|75|100re-renders the whole matrix at another tok/s/user target (default 50). The switcher is plain links, no client JS, so every view is a shareable URL.FP4 vs FP8 · no comparable delta,Different releases · no comparable delta.∞plus its reason (no 8K/1K data,cannot reach @50,standard decode only, …). There is never a percentage against a missing number.At 100, X leads.Testing
overview-data.test.ts: per-platform bucket selection, tier parameterization, mismatch and missing-reason classification. Expected values come from running the real builder — tier values are spline-interpolated, so nothing is hand-computed.overview.cy.tscovers the matrix, the tier links, mismatch and∞states, the Chinese sibling, and 390/320px overflow, on Chrome and Firefox.中文说明
新增
/overview与/zh/overview：服务端渲染的对比页，回答一个问题——现在部署这个模型，各平台能交付多少容量。框架 · 精度，完整配置在链接悬浮提示中。对比对象是完整服务栈，不是单独的芯片，页面方法说明中已写明。?tier=30|75|100切换服务档位（默认 50）。切换器是纯链接，无客户端 JS，任何视图都是可转发的 URL。∞与原因，绝不对缺失数字给出百分比。Note
Medium Risk
New public benchmark comparison surface with non-trivial ranking, interpolation, and delta rules; mistakes would misstate platform performance, though behavior is heavily unit- and e2e-tested.
Overview
Adds
/overviewand/zh/overview: a force-dynamic, server-rendered matrix of active models vs B200, MI355X, B300, GB200, and GB300 on fixed 8K→1K speculative-decode workloads. A newoverview-datapipeline (withoverview-config-identity, extendedtco-feedevidence_date, andgetOverviewPageData) picks each platform’s best stack per tier, computes B200 deltas only for same precision and release, and surfaces gaps as∞with explicit reasons.overview-scorecardrenders desktop and mobile from shared cell components; plain?tier=links andoverview-linksdeep-link to filtered/inferenceviews (full page loads for model detail).Also registers the route in i18n mirrors and sitemap, extracts
getCachedBenchmarkstobenchmark-data.server, movescomputeToggletotoggle-set, and tightens the header for narrow viewports (44px targets, hide GitHub stars belowsm, Minecraft audio toggles in the hamburger). Cypress e2e and Vitest coverage lock matrix semantics, tiers, zh copy, and fixture drift.Reviewed by Cursor Bugbot for commit 7378a3b. Bugbot is set up for automated code reviews on this repo. Configure here.