Skip to content

Commit ae36740

Browse files
authored
feat(agentic): Agent X cosmetic improvements / Agent X 外观改进 (#607)
* feat(agentic): enrich benchmark point tooltips Preserve KV offload backend versions and multinode transfer engines through ingest, then show offload type, backend/version, transfer engine, and cache-hit rates across official, comparison, and unofficial-run tooltips. 中文:丰富 Agent X 基准测试数据点的提示卡信息。在摄取流程中保留 KV 卸载后端版本和多节点传输引擎,并在正式运行、GPU 对比及非正式运行覆盖层的提示卡中展示卸载类型、后端及版本、传输引擎和 Cache 命中率。 * fix(inference): restore unofficial overlay tooltips Pass the portaled tooltip element through the shared D3 render context so unofficial-run points can render and pin the same tooltip as official points. 中文:通过共享 D3 渲染上下文传递 Portal 中的提示卡节点,使非正式运行数据点能够与正式运行数据点一样显示并固定提示卡。 * feat(agentic): surface runtime component metadata Preserve and render KV offload engines, KV transfer engines, and router names/versions across official, comparison, fixed-sequence, agentic, and unofficial-run tooltips. Add an artifact-backed metadata backfill that merges only runtime keys into historical rows. 中文:展示 Agentic 运行时组件元数据。在正式运行、GPU 对比、固定序列长度、Agentic 及非正式运行提示卡中保留并展示 KV 卸载引擎、KV 传输引擎以及路由器名称和版本;新增基于原始产物的元数据回填,仅向历史数据合并运行时字段。 * fix(inference): preserve legacy and localized tooltips Keep an explicit legacy offload fallback when detailed runtime metadata is unavailable, and rebuild unofficial overlay layers when the locale changes. 中文:保留旧版及本地化提示卡。在缺少详细运行时元数据时显示明确的旧版卸载状态,并在语言切换后重新构建非正式运行覆盖层。 * fix(inference): scope legacy offload fallback Show the binary fallback only for legacy agentic rows so fixed-sequence database defaults do not create misleading tooltip metadata. 中文:限定旧版卸载状态的适用范围。仅对旧版 Agentic 数据显示二元回退信息,避免固定序列长度数据的数据库默认值产生误导性提示卡。 * fix(db): harden runtime metadata backfill Store component versions only when their names are valid and include the producer rollout date itself in historical backfill candidates. 中文:强化运行时元数据回填。仅在组件名称有效时保留对应版本,并将生产端上线当天纳入历史回填候选范围。 * fix(agentic): hide CPU cache hits without offload Gate CPU Cache hit-rate rows behind the canonical KV-offload state in shared point tooltips and the agentic detail summary. Preserve GPU and theoretical rates, legacy fallback behavior, and unofficial-run overlays.\n\n中文:仅在启用 KV 卸载时展示 CPU Cache 命中率,统一覆盖数据点提示卡和 Agentic 详情摘要;保留 GPU 与理论命中率、旧版数据回退逻辑以及非正式运行覆盖层支持。 * fix(inference): omit disabled offload type Hide the Offload Type row when the canonical KV-offloading descriptor is none while preserving non-empty types and legacy agentic fallbacks. 中文:当规范化 KV 卸载描述为 none 时隐藏“卸载类型”行,同时保留有效卸载类型和旧版 Agentic 数据的回退展示。
1 parent 3959bff commit ae36740

24 files changed

Lines changed: 821 additions & 27 deletions

packages/app/cypress/e2e/agentic-point-time-series.cy.ts

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -222,7 +222,9 @@ const pointMeta = {
222222
date: '2026-06-23',
223223
run_url: null,
224224
server_gpu_cache_hit_rate: 0.5,
225-
server_cpu_cache_hit_rate: null,
225+
// Reproduces historical rows that retained a CPU hit metric after offload
226+
// was disabled. The agentic point summary must suppress this stale value.
227+
server_cpu_cache_hit_rate: 0.42,
226228
};
227229

228230
const sourceSeries = (source: Record<string, unknown>, prompt: number, generation: number) => ({
@@ -297,6 +299,9 @@ describe('Agentic point orchestrator metric sources', () => {
297299
});
298300

299301
it('switches every server chart to an orchestrator-normalized worker', () => {
302+
cy.contains('GPU cache hit').should('be.visible');
303+
cy.contains('CPU cache hit').should('not.exist');
304+
300305
cy.get('[data-testid="metric-source-toolbar"]')
301306
.should('have.css', 'position', 'sticky')
302307
.and('have.css', 'top', '64px');

packages/app/cypress/e2e/gpu-compare-agentic-detail.cy.ts

Lines changed: 16 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -108,10 +108,18 @@ const agenticBenchmarks = AGENTIC_HARDWARE.flatMap((g) =>
108108
isl: null,
109109
osl: null,
110110
conc,
111-
offload_mode: 'off',
111+
offload_mode: 'on',
112112
benchmark_type: 'agentic_traces',
113113
image: 'vllm/vllm-openai:v0.9.0',
114-
metrics: agenticMetrics(conc),
114+
metrics: {
115+
...agenticMetrics(conc),
116+
kv_offloading: 'dram',
117+
kv_offload_backend: 'mooncake',
118+
kv_offload_backend_version: '0.3.11.post1',
119+
router_name: 'vllm-router',
120+
router_version: '0.1.14',
121+
server_gpu_cache_hit_rate: 0.875,
122+
},
115123
workers: null,
116124
date: AGENTIC_DATE,
117125
run_url: null,
@@ -177,6 +185,12 @@ describe('GPU comparison agentic point detail', () => {
177185
});
178186

179187
cy.get('[data-chart-tooltip]:visible').should('have.length', 1);
188+
cy.get('[data-chart-tooltip]:visible')
189+
.should('contain', 'Offload Type: DRAM')
190+
.and('contain', 'KV Offload Engine: Mooncake 0.3.11.post1')
191+
.and('contain', 'Router: vLLM Router 0.1.14')
192+
.and('contain', 'GPU Cache Hit Rate: 87.5%')
193+
.and('not.contain', 'Offload Mode');
180194
cy.get('[data-chart-tooltip]:visible [data-action="view-charts"]')
181195
.should('be.visible')
182196
.then(($link) => {

packages/app/cypress/e2e/unofficial-watermark.cy.ts

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,16 @@ describe('Unofficial-run watermark', () => {
1919
],
2020
benchmarks: benchmarks.map((row: Record<string, unknown>) => ({
2121
...row,
22+
is_multinode: true,
23+
metrics: {
24+
...(row.metrics as Record<string, unknown> | undefined),
25+
kv_offloading: 'dram',
26+
kv_offload_backend: 'hicache',
27+
kv_p2p_transfer: 'nixl',
28+
router_name: 'sglang-router',
29+
router_version: '0.3.2',
30+
server_gpu_cache_hit_rate: 0.875,
31+
},
2232
run_url: runUrl,
2333
})),
2434
evaluations: [],
@@ -58,6 +68,20 @@ describe('Unofficial-run watermark', () => {
5868
cy.wrap($image).parent('svg').find('.unofficial-watermark-image').should('have.length', 1);
5969
});
6070

71+
cy.get('[data-testid="scatter-graph"] .unofficial-overlay-pt')
72+
.first()
73+
.then(($point) => {
74+
$point[0].dispatchEvent(new MouseEvent('mouseenter'));
75+
const tooltip = $point[0].ownerDocument.querySelector<HTMLElement>('[data-chart-tooltip]');
76+
expect(tooltip).not.to.equal(null);
77+
expect(tooltip!.style.display).to.equal('block');
78+
expect(tooltip).to.contain.text('Offload Type: DRAM');
79+
expect(tooltip).to.contain.text('KV Offload Engine: HiCache');
80+
expect(tooltip).to.contain.text('KV Transfer Engine: NIXL');
81+
expect(tooltip).to.contain.text('Router: SGLang Router 0.3.2');
82+
expect(tooltip).to.contain.text('GPU Cache Hit Rate: 87.5%');
83+
});
84+
6185
cy.get('[data-testid="scatter-graph"]').first().scrollIntoView();
6286
cy.screenshot('unofficial-watermark', { capture: 'viewport' });
6387

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
import { createElement } from 'react';
2+
import { renderToStaticMarkup } from 'react-dom/server';
3+
import { describe, expect, it } from 'vitest';
4+
5+
import type { PointMeta } from '@/hooks/api/use-trace-server-metrics';
6+
7+
import { PointSummary } from './point-summary';
8+
9+
function meta(overrides: Partial<PointMeta> = {}): PointMeta {
10+
return {
11+
id: 206885,
12+
hardware: 'gb200',
13+
framework: 'dynamo-vllm',
14+
model: 'deepseek-r1-0528',
15+
precision: 'fp8',
16+
spec_method: 'none',
17+
disagg: true,
18+
conc: 128,
19+
offload_mode: 'off',
20+
isl: null,
21+
osl: null,
22+
benchmark_type: 'agentic_traces',
23+
date: '2026-06-23',
24+
run_url: null,
25+
server_gpu_cache_hit_rate: 0.5,
26+
server_cpu_cache_hit_rate: 0.42,
27+
...overrides,
28+
};
29+
}
30+
31+
describe('PointSummary', () => {
32+
it('hides a stale CPU cache hit rate when offload is disabled', () => {
33+
const html = renderToStaticMarkup(createElement(PointSummary, { meta: meta() }));
34+
35+
expect(html).toContain('GPU cache hit');
36+
expect(html).not.toContain('CPU cache hit');
37+
});
38+
39+
it('shows the CPU cache hit rate when offload is enabled', () => {
40+
const html = renderToStaticMarkup(
41+
createElement(PointSummary, { meta: meta({ offload_mode: 'on' }) }),
42+
);
43+
44+
expect(html).toContain('CPU cache hit');
45+
expect(html).toContain('42.00%');
46+
});
47+
});

packages/app/src/components/inference/agentic-point/point-summary.tsx

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@
33
import type { ReactNode } from 'react';
44

55
import type { PointMeta } from '@/hooks/api/use-trace-server-metrics';
6+
import { isKvOffloadEnabled } from '@/lib/kv-offload';
67

78
const fmtPct = (v: number | null | undefined): string =>
89
v === null || v === undefined || Number.isNaN(v) ? '—' : `${(v * 100).toFixed(2)}%`;
@@ -18,6 +19,8 @@ function MetaLine({ label, value }: { label: string; value: ReactNode }) {
1819

1920
/** Selected-point header: config facts (offload, concurrency, cache hit rates, ISL/OSL). */
2021
export function PointSummary({ meta }: { meta: PointMeta }) {
22+
const showCpuCacheHit = isKvOffloadEnabled(meta);
23+
2124
return (
2225
<div className="mb-4">
2326
<div className="flex items-baseline justify-between gap-3 mb-2">
@@ -41,7 +44,9 @@ export function PointSummary({ meta }: { meta: PointMeta }) {
4144
<MetaLine label="Offload" value={(meta.offload_mode ?? 'off').toUpperCase()} />
4245
<MetaLine label="Concurrency" value={meta.conc} />
4346
<MetaLine label="GPU cache hit" value={fmtPct(meta.server_gpu_cache_hit_rate)} />
44-
<MetaLine label="CPU cache hit" value={fmtPct(meta.server_cpu_cache_hit_rate)} />
47+
{showCpuCacheHit && (
48+
<MetaLine label="CPU cache hit" value={fmtPct(meta.server_cpu_cache_hit_rate)} />
49+
)}
4550
{meta.isl !== null && <MetaLine label="ISL" value={meta.isl} />}
4651
{meta.osl !== null && <MetaLine label="OSL" value={meta.osl} />}
4752
</div>

packages/app/src/components/inference/types.ts

Lines changed: 13 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -190,9 +190,21 @@ export interface AggDataEntry {
190190
isl?: number | null;
191191
/** OSL in tokens — null for agentic_traces. */
192192
osl?: number | null;
193-
// ── Agentic-only fields (populated from metrics JSONB for `agentic_traces` rows) ──
193+
// ── Runtime cache metadata (populated from metrics JSONB when emitted) ──
194194
/** "on" | "off" — whether KV cache offload to CPU was enabled. */
195195
offload_mode?: string;
196+
/** Offload tier/type, for example `dram` or `none`. */
197+
kv_offloading?: string;
198+
/** Offload implementation, for example `mooncake`, `lmcache`, or `hicache`. */
199+
kv_offload_backend?: string;
200+
/** Optional version independently declared for the offload backend. */
201+
kv_offload_backend_version?: string;
202+
/** P2P engine used to move KV state between workers on multinode runs. */
203+
kv_p2p_transfer?: string;
204+
/** Request router implementation, for example `vllm-router` or `sglang-router`. */
205+
router_name?: string;
206+
/** Version independently declared for the request router. */
207+
router_version?: string;
196208
/** Actual server-observed GPU prefix-cache hit rate (0..1). */
197209
server_gpu_cache_hit_rate?: number;
198210
/** Actual server-observed CPU prefix-cache hit rate (0..1). */

packages/app/src/components/inference/ui/GPUGraph.tsx

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -853,6 +853,7 @@ const GPUGraph = React.memo(
853853
hardwareConfig,
854854
runUrl: d.run_url ? updateRepoUrl(d.run_url) : undefined,
855855
hasTrace: typeof d.id === 'number' ? traceAvailability?.[d.id] === true : false,
856+
locale,
856857
}),
857858
getRulerX: (d, xScale) => (xScale as d3.ScaleLinear<number, number>)(d.x),
858859
getRulerY: (d, yScale) => (yScale as d3.ScaleLinear<number, number>)(d.y),

packages/app/src/components/inference/ui/ScatterGraph.tsx

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1204,6 +1204,7 @@ const ScatterGraph = React.memo(
12041204
isTracked: trackedConfigIdsRef.current.has(buildPointConfigId(d)),
12051205
runUrl: d.run_url ? updateRepoUrl(d.run_url) : undefined,
12061206
hasTrace: typeof d.id === 'number' ? traceAvailability?.[d.id] === true : false,
1207+
locale,
12071208
}),
12081209
getRulerX: (d: InferenceData, xScale: any) => (xScale as ContinuousScale)(d.x),
12091210
getRulerY: (d: InferenceData, yScale: any) => (yScale as ContinuousScale)(d.y),
@@ -1270,6 +1271,7 @@ const ScatterGraph = React.memo(
12701271
// tooltip content closure (the "View charts" button), so rebuild the
12711272
// config when the presence fetch resolves.
12721273
traceAvailability,
1274+
locale,
12731275
],
12741276
);
12751277

@@ -2211,8 +2213,7 @@ const ScatterGraph = React.memo(
22112213
// Overlay tooltip handlers
22122214
const svgNode = ctx.layout.svg.node()!;
22132215
const container = svgNode.parentElement as HTMLDivElement;
2214-
const tooltipDiv = svgNode.nextElementSibling as HTMLDivElement;
2215-
const tooltip = d3.select(tooltipDiv);
2216+
const tooltip = d3.select(ctx.tooltipElement);
22162217

22172218
const createOverlayConfig = (d: InferenceData, pinned: boolean) => ({
22182219
data: d,
@@ -2222,6 +2223,7 @@ const ScatterGraph = React.memo(
22222223
selectedYAxisMetric,
22232224
hardwareConfig: overlayData.hardwareConfig,
22242225
overlayData,
2226+
locale,
22252227
});
22262228

22272229
overlayPoints
@@ -2493,6 +2495,7 @@ const ScatterGraph = React.memo(
24932495
yLabel,
24942496
selectedYAxisMetric,
24952497
chartDefinition,
2498+
locale,
24962499
]);
24972500

24982501
// Layers handle for the decoration effect — lets it re-run individual

0 commit comments

Comments
 (0)