Skip to content

fix(models): replace retired Anthropic model ids with pinned live models#2583

Closed
jucor wants to merge 1 commit into
stablefrom
jc/pin-anthropic-models-stable
Closed

fix(models): replace retired Anthropic model ids with pinned live models#2583
jucor wants to merge 1 commit into
stablefrom
jc/pin-anthropic-models-stable

Conversation

@jucor

@jucor jucor commented Jul 4, 2026

Copy link
Copy Markdown
Collaborator

Problem

Three Anthropic model IDs were hardcoded across the codebase and have since been retired by Anthropic — each now returns HTTP 404 from the Models API (verified against the live API on 2026-07-04), silently breaking every LLM path that falls through to them as a default:

Retired ID Retired Used in
claude-3-7-sonnet-20250219 2026-02-19 server: reportNarrative.ts, routes/delphi/jobs.ts, report_experimental/scripts/sonnet.ts
claude-opus-4-20250514 2026-06-15 server: collectiveStatement.ts; client-report: CommentsReport.jsx
claude-3-5-sonnet-20241022 2025-10-28 delphi: batch-results fallback default + advertised model list

The report client (CommentsReport.jsx) posts claude-opus-4-20250514 to /api/v3/delphi/batchReports and the FULL_PIPELINE job form, so the report UI's "generate report" / "generate narrative report" actions ship a dead model into the delphi pipeline.

Fix

Replace with the current pinned models. This generation has no dated-snapshot IDs — the versioned name is itself the immutable pin (the API rejects any date suffix for this generation; verified 2026-07-04):

  • Opus tier → claude-opus-4-8 — collective statements; report-pipeline job payloads
  • Sonnet tier → claude-sonnet-5 — narratives; experimental report; batch fallback

Each site keeps its existing tier; overridable defaults stay overridable (only the string literal changed). The three delphi docs that documented a retired ID as a default/example are updated to match.

Guard tests

Adds a guard test in each affected ecosystem — delphi (pytest), server (jest), client-report (jest) — that fails if any source references a retired model ID, so this class of breakage cannot silently regress.

Verification

  • All three retired IDs → HTTP 404; both replacements → 200 (live Models API, 2026-07-04).
  • delphi guard: RED (2 failed) → GREEN (2 passed) via uv run pytest.
  • server + client-report guards: RED → GREEN witnessed.
  • Independent code-review agent: no blocking issues (one guard-test efficiency finding — a scan walking the local .venv — was found and fixed).
  • Commits are GPG-signed.

Paired change: this PR targets stable; a companion PR ports the identical fix to edge (the retired-ID footprint is identical on both branches).

🤖 Generated with Claude Code

@jucor

jucor commented Jul 4, 2026

Copy link
Copy Markdown
Collaborator Author

Edge port: #2584

Three Anthropic model ids were hardcoded across the codebase and have since
been retired by Anthropic; each now returns HTTP 404 from the Models API,
silently breaking every LLM code path that falls through to them:

  claude-3-7-sonnet-20250219   (retired 2026-02-19)
  claude-opus-4-20250514       (retired 2026-06-15)
  claude-3-5-sonnet-20241022   (retired 2025-10-28)

Replace them with the current pinned models. The current generation has no
dated-snapshot ids — the versioned name is itself the immutable pin (verified
against the live Models API on 2026-07-04; the API rejects any date suffix for
this generation):

  Opus tier   -> claude-opus-4-8   (collective statements; report pipeline jobs)
  Sonnet tier -> claude-sonnet-5   (narratives; experimental report; batch fallback)

Sites fixed:
  server        collectiveStatement.ts, reportNarrative.ts, routes/delphi/jobs.ts,
                prompts/report_experimental/scripts/sonnet.ts
  client-report CommentsReport.jsx (report + narrative generation request payloads
                posted to the delphi pipeline)
  delphi        AnthropicProvider.list_available_models, 802 batch fallback default,
                and the JOB_QUEUE_SCHEMA / BATCH_NARRATIVE_README / DATA_FORMAT_STANDARDS
                docs that documented the retired ids as defaults/examples

Add guard tests (delphi pytest, server jest, client-report jest) that fail if
any source references a retired model id, so this class of breakage cannot
silently regress.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@jucor
jucor force-pushed the jc/pin-anthropic-models-stable branch from b1dbb64 to ceced28 Compare July 4, 2026 16:35
@github-actions

github-actions Bot commented Jul 4, 2026

Copy link
Copy Markdown

Delphi Coverage Report

File Stmts Miss Cover
init.py 2 0 100%
benchmarks/bench_pca.py 76 76 0%
benchmarks/bench_repness.py 81 81 0%
benchmarks/bench_update_votes.py 38 38 0%
benchmarks/benchmark_utils.py 34 34 0%
components/init.py 1 0 100%
components/config.py 165 133 19%
conversation/init.py 2 0 100%
conversation/conversation.py 1062 296 72%
conversation/manager.py 131 42 68%
database/init.py 1 0 100%
database/dynamodb.py 387 234 40%
database/postgres.py 306 206 33%
pca_kmeans_rep/init.py 5 0 100%
pca_kmeans_rep/clusters.py 257 22 91%
pca_kmeans_rep/corr.py 98 17 83%
pca_kmeans_rep/pca.py 52 16 69%
pca_kmeans_rep/repness.py 298 35 88%
regression/init.py 4 0 100%
regression/clojure_comparer.py 188 20 89%
regression/comparer.py 887 720 19%
regression/datasets.py 135 27 80%
regression/recorder.py 36 27 25%
regression/utils.py 138 94 32%
run_math_pipeline.py 261 114 56%
umap_narrative/500_generate_embedding_umap_cluster.py 210 109 48%
umap_narrative/501_calculate_comment_extremity.py 112 53 53%
umap_narrative/502_calculate_priorities.py 135 135 0%
umap_narrative/700_datamapplot_for_layer.py 502 502 0%
umap_narrative/701_static_datamapplot_for_layer.py 310 310 0%
umap_narrative/702_consensus_divisive_datamapplot.py 432 432 0%
umap_narrative/801_narrative_report_batch.py 785 785 0%
umap_narrative/802_process_batch_results.py 265 265 0%
umap_narrative/803_check_batch_status.py 175 175 0%
umap_narrative/llm_factory_constructor/init.py 2 0 100%
umap_narrative/llm_factory_constructor/model_provider.py 157 124 21%
umap_narrative/polismath_commentgraph/init.py 1 0 100%
umap_narrative/polismath_commentgraph/cli.py 270 270 0%
umap_narrative/polismath_commentgraph/core/init.py 3 3 0%
umap_narrative/polismath_commentgraph/core/clustering.py 108 108 0%
umap_narrative/polismath_commentgraph/core/embedding.py 104 104 0%
umap_narrative/polismath_commentgraph/lambda_handler.py 219 219 0%
umap_narrative/polismath_commentgraph/schemas/init.py 2 0 100%
umap_narrative/polismath_commentgraph/schemas/dynamo_models.py 160 9 94%
umap_narrative/polismath_commentgraph/tests/conftest.py 17 17 0%
umap_narrative/polismath_commentgraph/tests/test_clustering.py 74 74 0%
umap_narrative/polismath_commentgraph/tests/test_embedding.py 55 55 0%
umap_narrative/polismath_commentgraph/tests/test_storage.py 87 87 0%
umap_narrative/polismath_commentgraph/utils/init.py 3 0 100%
umap_narrative/polismath_commentgraph/utils/converter.py 283 237 16%
umap_narrative/polismath_commentgraph/utils/group_data.py 354 336 5%
umap_narrative/polismath_commentgraph/utils/storage.py 584 518 11%
umap_narrative/reset_conversation.py 159 50 69%
umap_narrative/run_pipeline.py 453 312 31%
utils/general.py 62 41 34%
Total 10728 7562 30%

@jucor

jucor commented Jul 4, 2026

Copy link
Copy Markdown
Collaborator Author

@ballPointPenguin :)

@jucor

jucor commented Jul 4, 2026

Copy link
Copy Markdown
Collaborator Author

Closed as discussed with @ballPointPenguin , in favour of a single PR #2584 on edge, since stable is at parity with edge at the moment.

@jucor jucor closed this Jul 4, 2026
@jucor
jucor deleted the jc/pin-anthropic-models-stable branch July 12, 2026 16:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant