Skip to content

Commit 67c731e

Browse files
committed
chore(metadata): resync jawcode bundle, re-block hy3-preview, add a drift guard
The committed generated file had drifted 95 models from its source — 148 price fields, 45 maxTokens, 36 context windows, 36 new models — because nothing ever checked it. That made any regeneration a blind bulk change, so this lands the resync with each class of delta accounted for. Price movements, classified rather than trusted: - x1.1 exactly on the amazon-bedrock `eu.*` rows — the AWS EU regional premium (1 -> 1.1, 5 -> 5.5, 25 -> 27.5), plus one row moving the other way. - ~x1.0-1.3 across ~40 openrouter rows; that bundle tracks live marketplace pricing, so drift there is the expected steady state. - The two outliers are aliases, not repricing: `gemini-flash-latest` (x5) and `gemini-flash-lite-latest` (x2.5) were repointed from 2.5-flash to 3.5-flash. Confirmed the new values are byte-identical to the `gemini-3.5-flash` row, which is what a `-latest` alias is supposed to do. MiniMax-M3 picks up `video` here as the clean one-row delta it was always meant to be (jawcode `762549f`), with its cost row untouched. **`hy3-preview` is re-blocked, and this is the important part.** Upstream re-added `tencent/hy3-preview`, so the regeneration resurrected `opencode-go/hy3-preview` — the model from lidge-jun#82 that returns `Provider error 400 … model_not_supported` because it is not on the Zen Go lite list. lidge-jun#82 was only ever "fixed" by upstream absence; there was no filter. There is one now (`EXCLUDED_MODELS`), so a future re-add cannot hand users that 400 again. `hy3` itself stays selectable. Two stale expectations updated with their reasons: - `claude-opus-5` now has a real `anthropic` jawcode row at the *same* 5/25/0.5/6.25 the maintainer derived, so its provenance moved `expected/verified-derived` -> `jawcode/verified` for that provider only. cursor and kiro still resolve through the overlay, so the overlay stays. - `opencode-go` contexts follow upstream: `qwen3.5-plus` to 1M, `minimax-m3` down to the 512K tier that matches MiniMax's own ≤512K band. The new guard regenerates into a temp file and byte-compares, so it cannot clobber the committed output. It skips when the jawcode checkout is absent — the generator throws without it, so asserting unconditionally would redden every contributor and CI run, and drift can only be introduced by someone who has the source anyway. Verified both directions: passes in sync, fails on a one-field perturbation. (`JAWCODE_METADATA_OUT` exists to make that safe.) Gates: tsc clean; sync-guard + codex-catalog + usage-cost + parity + slug-codec + minimax-reasoning-split 203 pass / 1 skip / 0 fail; privacy scan passed.
1 parent ef805c3 commit 67c731e

5 files changed

Lines changed: 104 additions & 20 deletions

File tree

scripts/generate-jawcode-metadata.ts

Lines changed: 15 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,11 @@ type RawModel = {
2222
const sourcePath = process.env.JAWCODE_MODELS_JSON
2323
? resolve(process.env.JAWCODE_MODELS_JSON)
2424
: resolve(process.cwd(), "../jawcode/packages/ai/src/models.json");
25-
const outPath = resolve(process.cwd(), "src/generated/jawcode-model-metadata.ts");
25+
// `JAWCODE_METADATA_OUT` lets the sync guard regenerate into a temp file and byte-compare without
26+
// any risk of clobbering the committed output (tests/jawcode-metadata-sync.test.ts).
27+
const outPath = process.env.JAWCODE_METADATA_OUT
28+
? resolve(process.env.JAWCODE_METADATA_OUT)
29+
: resolve(process.cwd(), "src/generated/jawcode-model-metadata.ts");
2630
const CONTEXT_WINDOW_OVERRIDES: Record<string, number> = {
2731
"anthropic/claude-sonnet-4-6": 200_000,
2832
"anthropic/claude-sonnet-4-6[1m]": 200_000,
@@ -45,6 +49,15 @@ const COST_VENDOR_BUNDLES = [
4549
const allowedProviders = Array.from(
4650
new Set([...Object.values(PROVIDER_ALIASES), ...COST_VENDOR_BUNDLES]),
4751
).sort();
52+
53+
// Models the upstream catalogue lists but the provider rejects at request time. Issue #82:
54+
// `opencode-go/hy3-preview` returns `Provider error 400 ... model_not_supported` because it is not
55+
// on the Zen Go lite model list, yet upstream keeps re-adding `tencent/hy3-preview`. Without this
56+
// block, every regeneration resurrects it as a selectable model and users hit that 400 again.
57+
// Keyed as `<bundle>/<modelId>`; `tests/codex-catalog.test.ts` asserts the slug stays absent.
58+
const EXCLUDED_MODELS = new Set<string>([
59+
"opencode-go/hy3-preview",
60+
]);
4861
const lines: string[] = [];
4962

5063
function compactRow(values: unknown[]): unknown[] {
@@ -74,6 +87,7 @@ lines.push("const DATA: Record<string, readonly Row[]> = {");
7487
for (const provider of allowedProviders) {
7588
const models = registry[provider] ?? {};
7689
const rows = Object.entries(models)
90+
.filter(([id]) => !EXCLUDED_MODELS.has(`${provider}/${id}`))
7791
.sort(([a], [b]) => a.localeCompare(b))
7892
.map(([id, model]) => compactRow([
7993
id,

src/generated/jawcode-model-metadata.ts

Lines changed: 12 additions & 12 deletions
Large diffs are not rendered by default.

tests/codex-catalog.test.ts

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2071,9 +2071,11 @@ describe("Codex catalog routed normalization", () => {
20712071
test("opencode-go high-risk models use official jawcode metadata in the Codex catalog", () => {
20722072
const cases = [
20732073
{ id: "glm-5.2", context: 1_000_000, auto: 900_000, input: ["text"] },
2074-
{ id: "qwen3.5-plus", context: 262_144, auto: 235_929, input: ["text", "image"] },
2074+
// Upstream raised qwen3.5-plus to a 1M window and cut minimax-m3 to the 512K tier that
2075+
// matches MiniMax's own <=512K pricing band; both are catalogue refreshes, not overrides.
2076+
{ id: "qwen3.5-plus", context: 1_000_000, auto: 900_000, input: ["text", "image"] },
20752077
{ id: "kimi-k2.7-code", context: 262_144, auto: 235_929, input: ["text", "image"] },
2076-
{ id: "minimax-m3", context: 1_000_000, auto: 900_000, input: ["text", "image"] },
2078+
{ id: "minimax-m3", context: 512_000, auto: 460_800, input: ["text", "image"] },
20772079
{ id: "qwen3.7-max", context: 1_000_000, auto: 900_000, input: ["text"] },
20782080
] as const;
20792081
const entries = buildCatalogEntries(nativeTemplate(), [], cases.map(({ id }) => ({ provider: "opencode-go", id })));
Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
import { describe, expect, test } from "bun:test";
2+
import { existsSync, mkdtempSync, readFileSync } from "node:fs";
3+
import { tmpdir } from "node:os";
4+
import { join, resolve } from "node:path";
5+
6+
/**
7+
* `src/generated/jawcode-model-metadata.ts` is generated from the jawcode `models.json`, and until
8+
* now nothing checked that the committed file still matched its source. It drifted by 95 models —
9+
* 148 price fields, 36 context windows, 45 maxTokens, and 36 entirely new models — which meant a
10+
* routine regeneration would sweep all of that into cost accounting alongside whatever the author
11+
* actually intended to change.
12+
*
13+
* This guard closes that. It regenerates into a temp directory and byte-compares, so it can never
14+
* clobber the committed file.
15+
*
16+
* It SKIPS when the jawcode checkout is absent, which is the normal case for contributors and for
17+
* CI: `scripts/generate-jawcode-metadata.ts` throws without it, so asserting unconditionally would
18+
* turn every external run red. The drift can only be introduced by someone who has the source, so
19+
* that is exactly where the check needs to fire.
20+
*/
21+
const GENERATED = resolve(import.meta.dir, "../src/generated/jawcode-model-metadata.ts");
22+
// Resolve the default exactly the way the generator does — relative to cwd, not to this file — so a
23+
// git worktree does not silently point at a sibling that happens to share the parent directory.
24+
const SOURCE = process.env.JAWCODE_MODELS_JSON
25+
? resolve(process.env.JAWCODE_MODELS_JSON)
26+
: resolve(process.cwd(), "../jawcode/packages/ai/src/models.json");
27+
28+
describe("generated jawcode metadata stays in sync with its source", () => {
29+
test.skipIf(!existsSync(SOURCE))(
30+
"regenerating reproduces the committed file byte for byte",
31+
async () => {
32+
const outDir = mkdtempSync(join(tmpdir(), "jawcode-sync-"));
33+
const outPath = join(outDir, "jawcode-model-metadata.ts");
34+
35+
const proc = Bun.spawn(
36+
["bun", resolve(import.meta.dir, "../scripts/generate-jawcode-metadata.ts")],
37+
{
38+
cwd: resolve(import.meta.dir, ".."),
39+
env: { ...process.env, JAWCODE_MODELS_JSON: SOURCE, JAWCODE_METADATA_OUT: outPath },
40+
stdout: "pipe",
41+
stderr: "pipe",
42+
},
43+
);
44+
const exitCode = await proc.exited;
45+
expect(exitCode, await new Response(proc.stderr).text()).toBe(0);
46+
47+
expect(readFileSync(outPath, "utf-8")).toBe(readFileSync(GENERATED, "utf-8"));
48+
},
49+
30_000,
50+
);
51+
});

tests/usage-cost.test.ts

Lines changed: 22 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -103,18 +103,35 @@ describe("normalizeCostTokens", () => {
103103
});
104104

105105
describe("resolveMatchedPrice", () => {
106-
// WP7: claude-opus-5 is exposed by three providers but missing from the jawcode
106+
// WP7: claude-opus-5 is exposed by three providers but was missing from the jawcode
107107
// bundle, so it resolved to null and Logs rendered an em dash instead of a cost.
108108
// The model-level vendor fallback only searches jawcode metadata, never overlays,
109-
// so each exposing provider needs its own row.
110-
test("claude-opus-5 resolves to the user-derived Opus 4.6 price on every exposing provider", () => {
111-
for (const provider of ["anthropic", "cursor", "kiro"]) {
109+
// so each exposing provider needed its own row.
110+
//
111+
// Upstream has since published an `anthropic/claude-opus-5` row at the SAME numbers the
112+
// maintainer derived (5 / 25 / 0.5 / 6.25), so that provider is now sourced from jawcode
113+
// and reads `verified` instead of `verified-derived`. cursor and kiro have no jawcode row
114+
// of their own and still come from the overlay, which is why the overlay must stay.
115+
// The price is identical either way — only the provenance moved, and it moved forward.
116+
test("claude-opus-5 resolves to the Opus 4.6 price on every exposing provider", () => {
117+
const COST4 = { input: 5, output: 25, cacheRead: 0.5, cacheWrite: 6.25 };
118+
119+
const anthropic = resolveMatchedPrice("anthropic", "claude-opus-5");
120+
expect(anthropic).toMatchObject({
121+
provider: "anthropic",
122+
modelId: "claude-opus-5",
123+
cost4: COST4,
124+
source: "jawcode",
125+
status: "verified",
126+
});
127+
128+
for (const provider of ["cursor", "kiro"]) {
112129
const price = resolveMatchedPrice(provider, "claude-opus-5");
113130
expect(price).not.toBeNull();
114131
expect(price).toMatchObject({
115132
provider,
116133
modelId: "claude-opus-5",
117-
cost4: { input: 5, output: 25, cacheRead: 0.5, cacheWrite: 6.25 },
134+
cost4: COST4,
118135
source: "expected",
119136
status: "verified-derived",
120137
});

0 commit comments

Comments
 (0)