fix(etl): map AMD Kimi-K2.7-Code identifiers to kimik2.7-code / 修复:将 AMD Kimi-K2.7-Code 标识映射到 kimik2.7-code#619
Merged
Conversation
AMD AgentX sweeps label the model with the bare prefix `kimik2.7` (no `-code`) and the MXFP4 model path `amd/Kimi-K2.7-Code-MXFP4`. Neither was in PREFIX_ALIASES or MODEL_TO_KEY, so resolveModelKey returned null and mapBenchmarkRow / the eval mapper skipped every row as `unmappedModel`. Effect: the unofficial-run overlay for such runs (e.g. GitHub Actions run 29975243963) fetched artifacts fine but produced 0 benchmarks and 0 evaluations, rendering nothing — and official ingest would drop the rows for the same reason. Add the bare prefix alias (which also covers precision-suffixed forms like `kimik2.7-fp4` via PRECISION_SUFFIX stripping) and the AMD MXFP4 model path, both folding into the canonical `kimik2.7-code` DB bucket. Consistent with existing `amd/…-MXFP4` entries (GLM-5.1, Llama-3.3-70B). 中文:AMD AgentX 基准测试(benchmark)的产物(artifact)用裸前缀 `kimik2.7` (不带 `-code`)和 MXFP4 模型路径 `amd/Kimi-K2.7-Code-MXFP4` 标注模型,二者都 不在 PREFIX_ALIASES / MODEL_TO_KEY 中,导致 resolveModelKey 返回 null,所有 行被当作未映射模型(unmappedModel)跳过。结果:此类运行的非官方运行(unofficial run)叠加层能正常拉取产物却解析出 0 条基准与 0 条评估(evaluation),页面空白; 官方摄取(ingest)也会因同样原因丢弃这些行。本次新增裸前缀别名(经 PRECISION_SUFFIX 去后缀后也覆盖 `kimik2.7-fp4` 等形式)与 AMD MXFP4 模型路径,统一归入规范的 `kimik2.7-code` 数据桶,与既有的 `amd/…-MXFP4` 条目(GLM-5.1、Llama-3.3-70B)保持一致。
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
There was a problem hiding this comment.
Pull request overview
This PR fixes a db-layer ETL normalization gap where AMD AgentX artifacts label Kimi-K2.7-Code using non-canonical identifiers, causing resolveModelKey to return null and downstream ingest/overlay pipelines to skip all rows as unmappedModel. It updates the shared normalizer mappings so those AMD identifiers fold into the existing canonical kimik2.7-code DB model key and adds a regression test to prevent reintroduction.
Changes:
- Add a
PREFIX_ALIASESentry mappingkimik2.7→kimik2.7-code(including precision-suffixed variants via existing suffix stripping). - Add a
MODEL_TO_KEYentry mappingamd/Kimi-K2.7-Code-MXFP4→kimik2.7-code. - Add a regression unit test covering both the bare/precision-suffixed prefixes and the AMD MXFP4 model path.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| packages/db/src/etl/normalizers.ts | Extends model-key normalization with AMD Kimi-K2.7-Code prefix/path aliases to resolve to kimik2.7-code. |
| packages/db/src/etl/normalizers.test.ts | Adds a regression test ensuring the new AMD identifiers resolve to the canonical key across both benchmark and eval row shapes. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
AMD AgentX sweeps label the model with the bare prefix
kimik2.7(no-code) and the MXFP4 model pathamd/Kimi-K2.7-Code-MXFP4. Neither was registered inPREFIX_ALIASESorMODEL_TO_KEY, soresolveModelKey(packages/db/src/etl/normalizers.ts) returnednulland bothmapBenchmarkRowand the eval mapper skipped every row asunmappedModel.Symptom: the unofficial-run overlay for such a run — e.g.
runs/29975243963([AMD][AgentX] Add Kimi-K2.7 MI355X vLLM agentic MTP) — fetched its artifacts fine but the/api/unofficial-runresponse contained 0 benchmarks and 0 evaluations, so?unofficialrun=…rendered nothing. Official ingest would drop the same rows for the same reason.I confirmed the root cause by running the real
resolveModelKeyagainst the exact values in the run's artifacts (infmax_model_prefix: "kimik2.7",model: "amd/Kimi-K2.7-Code-MXFP4", evalmodel_prefix: "kimik2.7") — all returnednull, while the canonicalkimik2.7-code/moonshotai/Kimi-K2.7-Coderesolved correctly.Fix
Two mapping additions in
normalizers.ts, both folding into the canonicalkimik2.7-codeDB bucket:PREFIX_ALIASES:'kimik2.7' → 'kimik2.7-code'— also covers precision-suffixed forms likekimik2.7-fp4via existingPRECISION_SUFFIXstripping.MODEL_TO_KEY:'amd/Kimi-K2.7-Code-MXFP4' → 'kimik2.7-code'— consistent with existingamd/…-MXFP4entries (amd/GLM-5.1-MXFP4,amd/Llama-3.3-70B-Instruct-MXFP4-Preview).Tests
Added a regression case to
resolveModelKeycovering the bare prefix, the precision-suffixed prefix, the evalmodel_prefix, the AMD MXFP4 path, and the canonical identifiers.pnpm test:unit(db package),pnpm lint,pnpm fmt,pnpm typecheckall pass.Scope / caveats
dbETL layer — no chart/UI code, so no overlay-path or/zhwork applies.29975243963itself worth ingesting officially: that run'sagentic evalgate failed withem_strict: 0.0on swebench_lite and only 552/1920 requests succeeded — its data is genuinely broken. The mapping fix simply ensures well-formed AMD K2.7-Code runs resolve going forward.中文说明
AMD AgentX 基准测试(benchmark)的产物(artifact)用裸前缀
kimik2.7(不带-code)和 MXFP4 模型路径amd/Kimi-K2.7-Code-MXFP4标注模型。二者都未在PREFIX_ALIASES或MODEL_TO_KEY中注册,导致resolveModelKey(packages/db/src/etl/normalizers.ts)返回null,mapBenchmarkRow与评估(evaluation)映射器把每一行都当作未映射模型(unmappedModel)跳过。现象: 此类运行的非官方运行(unofficial run)叠加层——例如
runs/29975243963——能正常拉取产物,但/api/unofficial-run返回 0 条基准、0 条评估,因此?unofficialrun=…页面空白;官方摄取(ingest)也会因同样原因丢弃这些行。我用运行产物中的真实取值调用resolveModelKey验证了根因:上述标识均返回null,而规范形式kimik2.7-code/moonshotai/Kimi-K2.7-Code能正确解析。修复: 在
normalizers.ts中新增两条映射,统一归入规范的kimik2.7-code数据桶:PREFIX_ALIASES:'kimik2.7' → 'kimik2.7-code'——经既有PRECISION_SUFFIX去后缀后,也覆盖kimik2.7-fp4等带精度后缀的形式。MODEL_TO_KEY:'amd/Kimi-K2.7-Code-MXFP4' → 'kimik2.7-code'——与既有amd/…-MXFP4条目(amd/GLM-5.1-MXFP4、amd/Llama-3.3-70B-Instruct-MXFP4-Preview)保持一致。测试: 为
resolveModelKey增加回归用例,覆盖裸前缀、带精度后缀的前缀、评估用model_prefix、AMD MXFP4 路径以及规范标识。pnpm test:unit(db 包)、pnpm lint、pnpm fmt、pnpm typecheck全部通过。范围/注意事项: 这是
dbETL 层的纯数据映射改动,不涉及图表/UI 代码,故不适用叠加层(overlay)路径或/zh相关工作。此修复解除了 AMD K2.7-Code 运行的渲染阻塞,但并不意味着29975243963这一运行值得官方摄取:该运行的agentic eval质量门以 swebench_liteem_strict: 0.0失败,且仅 552/1920 个请求成功,数据本身有问题。本修复只是保证今后格式正确的 AMD K2.7-Code 运行能够正常解析。Note
Low Risk
Scoped to static model-key aliases in the db ETL normalizers; no auth, API, or UI behavior changes beyond correctly ingesting/rendering rows that were previously skipped.
Overview
Fixes AMD AgentX Kimi-K2.7-Code runs being dropped as unmapped models during benchmark/eval ETL and unofficial-run overlays.
resolveModelKeynow maps the bare prefixkimik2.7(and precision-suffixed variants likekimik2.7-fp4) to the canonicalkimik2.7-codebucket viaPREFIX_ALIASES, and maps the model pathamd/Kimi-K2.7-Code-MXFP4viaMODEL_TO_KEY—matching other AMD MXFP4 entries.A regression test covers prefix, eval
model_prefix, AMD path, and existing canonical identifiers.Reviewed by Cursor Bugbot for commit f876a12. Bugbot is set up for automated code reviews on this repo. Configure here.