fix: refresh supported agent model catalog#1574
Open
simple-agent-manager[bot] wants to merge 9 commits into
Open
fix: refresh supported agent model catalog#1574simple-agent-manager[bot] wants to merge 9 commits into
simple-agent-manager[bot] wants to merge 9 commits into
Conversation
Contributor
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.



Summary
ModelGroup[]consumer contract.Validation
pnpm lint(existing warnings only)pnpm typecheckpnpm test(7,000+ tests)pnpm build; focused shared catalog tests 25/25; exact Models.dev ID/name comparisonStaging Verification (REQUIRED for all code changes — merge-blocking)
app.sammy.partyand passed all 12 Playwright smoke testsStaging Verification Evidence
The shared staging Worker had already been advanced from main's append-only Durable Object migration
v16to draft PR #1547'sv17. Deploying this branch directly therefore failed at Cloudflare migration reconciliation in run 29239922843. For safe verification, temporary branchsam/weekly-model-catalog-staging-zy3pn4was created from the branch owning livev17, and only catalog commitd6e668d62was cherry-picked.Final-head run 29243127194 passed deployment, D1 data-integrity checks, health checks, and 12/12 Playwright smoke tests after one retry for a transient Cloudflare
/zonesHTTP 500. Authenticated requests toapi.sammy.party/api/model-catalog/:agentTypematched this PR exactly for Claude (14 models), Codex (19), Gemini (6), and Mistral (10). The deployed OpenCode groups, IDs, and names had no diff from the current live Models.dev response.UI Compliance Checklist (Required for UI changes)
N/A: no UI code or layout changed; the existing model selector consumes the unchanged catalog contract.
End-to-End Verification (Required for multi-component changes)
N/A: this is a shared data refresh with an unchanged API/UI contract. Focused tests exercise shared catalog lookups and proxy invariants, and staging verified the authenticated API's final serialized responses plus the standard browser smoke suite.
Data Flow Trace
N/A: no multi-component control flow changed.
Untested Gaps
N/A: exact static responses and dynamic OpenCode source parity were verified on live staging.
Post-Mortem (Required for bug fix PRs)
What broke
The user-facing model catalog suggested retired or malformed model IDs and carried stale lifecycle labels; its OpenCode fallback had also drifted from Models.dev.
Root cause
Provider catalogs evolved after the prior static snapshot. This is cumulative external-catalog drift rather than a regression attributable to one repository commit.
Class of bug
External capability-catalog drift.
Why it wasn't caught
Existing tests protected catalog shape and selected invariants but did not assert all newly retired IDs, exact corrected provider IDs, or full fallback parity with the current source snapshot.
Process fix included in this PR
Expanded focused tests in
packages/shared/tests/model-catalog.test.tsandpackages/shared/tests/unit/model-catalog-alternative-providers.test.tscover critical exact IDs, removals, lifecycle labels, duplicates, active OpenCode entries, and proxy versus agent-only routing.Post-mortem file
tasks/archive/2026-07-13-refresh-supported-agent-model-catalog.mdSpecialist Review Evidence (Required for agent-authored PRs)
needs-human-reviewlabel added and merge deferred to humanExceptions (If any)
v17, requiring a temporary integration branch for a valid deployment; interrupted local reviewer reports trigger the mandatory human merge gateneeds-human-reviewremains until a human decides whether the locally repeated reviews are sufficientAgent Preflight (Required)
Classification
External References
Consulted official documentation before coding: Anthropic models, Codex models, Gemini CLI model selection, Mistral models, and the live Models.dev API. Additional lifecycle URLs and exact findings are recorded in the archived task file.
Codebase Impact Analysis
packages/shared/src/model-catalog.tsand its focused tests changed. Existing API and web consumers were traced and retain the sameModelGroup[]contract.Documentation & Specs
Updated
tasks/archive/2026-07-13-refresh-supported-agent-model-catalog.mdwith the dated research, current authoritative source URLs, implementation checklist, and acceptance criteria. No user guide or architecture document needed changes because theModelGroup[]interface and consumer behavior are unchanged.Constitution & Risk Check
Checked Principle XI (No Hardcoded Values). The modified identifiers and lifecycle metadata are the requested provider-owned static catalog data, not deployment configuration. Main risks were invalid IDs, retired suggestions, proxy-routing leakage, and source drift; exact tests and staging comparisons cover each.