Skip to content

feat(models): upgrade Kimi defaults to K3 - #9348

Open
scarmani wants to merge 8 commits into
mainfrom
codex/kimi-k3-upgrade-20260716
Open

feat(models): upgrade Kimi defaults to K3#9348
scarmani wants to merge 8 commits into
mainfrom
codex/kimi-k3-upgrade-20260716

Conversation

@scarmani

Copy link
Copy Markdown
Collaborator

Summary

  • add a canonical moonshotai/kimi-k3 OpenRouter pin and migrate active Kimi defaults, reviewer selection, PDB invocation/pricing, model metadata, UI labels, seed data, benchmarks, scripts, and current documentation
  • expose KimiK3Agent while retaining KimiK2Agent as an import compatibility alias
  • route kimi-thinking to K3 mandatory reasoning and make credential detection accurately require OpenRouter for current Kimi agents
  • retain explicit legacy K2/v1 aliases for stored configuration and pricing compatibility, the direct kimi-legacy API contract, and Factory Droid K2.5 because the installed Droid catalog does not expose K3

Availability proof

  • OpenRouter live catalog model: https://openrouter.ai/moonshotai/kimi-k3
  • live endpoint metadata reports the Moonshot-hosted moonshotai/kimi-k3 endpoint, 1,048,576-token context, text/image input, reasoning/tools support, and current pricing
  • no live inference was attempted because this environment has no OpenRouter/Moonshot provider key

Validation

  • 603 passed focused agent, credential, selector, PDB, model-pin, and quorum-evidence tests
  • Ruff and Ruff format
  • mypy on touched active K3 integration modules
  • TypeScript tsc --noEmit
  • docs consistency and capabilities YAML parse
  • python3 scripts/check_portability.py
  • git diff --check
  • bash scripts/automation_pr_preflight.sh origin/main HEAD
  • commit and push hooks

Governance

Draft because the change includes the Kimi model used by quorum-evidence review. No evidence, settlement, workflow rerun, or merge action is included.

@github-actions

Copy link
Copy Markdown
Contributor

OpenAPI Spec Update

The OpenAPI specification has changed. Please review the generated spec in the workflow artifacts.

@scarmani
scarmani marked this pull request as ready for review July 17, 2026 00:18
@scarmani
scarmani requested a review from an0mium as a code owner July 17, 2026 00:18
@github-actions

Copy link
Copy Markdown
Contributor

OpenAPI Spec Update

The OpenAPI specification has changed. Please review the generated spec in the workflow artifacts.

@github-actions

github-actions Bot commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

Aragora Code Review

Advisory-only review. No issues found.

Co-authored-by: codex[bot] <codex[bot]@users.noreply.github.com>
@github-actions

Copy link
Copy Markdown
Contributor

OpenAPI Spec Update

The OpenAPI specification has changed. Please review the generated spec in the workflow artifacts.

@scarmani

Copy link
Copy Markdown
Collaborator Author

OpenAI independent model review

Reviewer: openai (openai) — independent adversarial model review via Codex CLI OpenAI harness, grounded on the exact PR head.
Head: 84fb93c (84fb93c), committed 2026-07-17T01:00:59Z.
PR: #9348.
Model family: openai

Verdict: PASS

  • [P3] aragora/config/settings.py:1324 - kimi-legacy is added to the primary allowlist, but aragora.config.legacy.ALLOWED_AGENT_TYPES still omits it despite claiming to be the compatibility constant. Direct legacy import callers will see a different allowed-agent set.

dogfood: yes

@scarmani

Copy link
Copy Markdown
Collaborator Author

Grok independent model review

Reviewer: grok (xai) — independent adversarial model review via the Aragora Grok reviewer, grounded on the exact PR head.
Head: 84fb93c (84fb93c), committed 2026-07-17T01:00:59Z.
PR: #9348.
Model family: grok

Verdict: PASS

No findings.

dogfood: yes

@scarmani

Copy link
Copy Markdown
Collaborator Author

Tier-4 Human Settlement Authorization

PR: #9348
Exact head: 84fb93c
Authorized action: admin_squash_merge and branch_protection_reconcile, only if #9348 is non-draft and live exact-head checks/merge-packet remain otherwise green.

Human-risk settlement: I accept the Tier 4 risk for this PR.

@github-actions

Copy link
Copy Markdown
Contributor

OpenAPI Spec Update

The OpenAPI specification has changed. Please review the generated spec in the workflow artifacts.

scarmani added a commit that referenced this pull request Jul 17, 2026
…nce record

Quorum round-2 findings at 416fd85:
- P2 (claude): drop the kimi K3 lane flip — revert to moonshotai/kimi-k2.6.
  The reliability record itself defers K3 until live verification + a
  catalog entry exist (catalog work #9348 is unmerged). Test pin updated;
  doc notes the deferral explicitly.
- P2 (openai): the Tier-4 evidence artifact was gitignored. Committed
  verbatim as docs/governance/records/20260716T2200Z-gemini-reviewer-
  reliability-record.md with a provenance header; all citations
  (quorum_evidence.py, REVIEW_AUTHORITY_PRINCIPLES.md, governance test)
  repointed to the committed path, plus a test pinning its presence.
- P3 (openai): removed Google from the stale WESTERN_FAMILIES description
  comment.
- P3 (claude): doc now states Chinese-routed lanes fully count at Tier 0-2
  per the tier table (model-pin changes do not alter counting authority),
  and explicitly frames the reliability record + this PR's Tier-4
  settlement as an operator-approved substitute for the docs/specs/
  design-doc requirement, scoped to this demotion only.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

OpenAPI Spec Update

The OpenAPI specification has changed. Please review the generated spec in the workflow artifacts.

@github-actions

Copy link
Copy Markdown
Contributor

OpenAPI Spec Update

The OpenAPI specification has changed. Please review the generated spec in the workflow artifacts.

@scarmani

Copy link
Copy Markdown
Collaborator Author

Claude independent model review

Reviewer: claude (anthropic) — independent adversarial model review via the Aragora Claude reviewer, grounded on the exact PR head.
Head: 62e7c97 (62e7c97), committed 2026-07-18T02:22:57Z.
PR: #9348.
Model family: claude

Verdict: PASS

Reviewed the complete 42-file diff. This is a coordinated, well-tested kimi-k2*kimi-k3 rename with backward-compat aliasing (KimiK2Agent = KimiK3Agent, legacy model-id keys retained in fallback/pricing maps). Cross-file consistency is solid:

  • Agent-type additions (kimi-legacy) land in settings.py, legacy.py, base.py AgentType, and both allowlist tests — so the new agent_type="kimi-legacy" won't be rejected by server-side validation.
  • Credential/readiness maps agree: kimi/kimi-thinking → OpenRouter only, kimi-legacyKIMI_API_KEY; tests cover both directions.
  • Pricing is internally consistent: real_invoker kimi-k3 (3.00, 15.00)/MTok ↔ model_selector 0.003/0.015 per-1k; K3 key added so dispatch won't hit a missing-price path.
  • manifest.json and its generator write_manifest.py updated in lockstep (no regen drift).
  • The prior latent bug where KimiLegacyAgent instances reported agent_type="kimi" is fixed.

Findings:

  • [P3] aragora/agents/api_agents/openrouter.py ~L807-855: kimi and kimi-thinking now both resolve to the identical id moonshotai/kimi-k3 with no distinguishing reasoning parameter, yet the kimi-thinking description claims a "mandatory-reasoning mode." Nothing in code enables a distinct mode, so the two agent types are byte-identical — a heterogeneity regression from the prior separate kimi-k2-thinking model if a panel selects both. Intentional/documented as an alias, but the docstring overclaims a behavior that isn't wired.
  • [P3] Model slug moonshotai/kimi-k3 (release_date "2026-07", seed_agents.py) can't be verified against OpenRouter's live catalog from the diff. If the slug is wrong, callers get a 404/invalid-model error at runtime; the OPENROUTER_FALLBACK_MODELS safety net is triggered by quota/429 paths and may not cover an invalid-model error. Worth a one-time live confirmation before merge.

dogfood: yes

@scarmani

Copy link
Copy Markdown
Collaborator Author

OpenAI independent model review

Reviewer: openai (openai) — independent adversarial model review via Codex CLI OpenAI harness, grounded on the exact PR head.
Head: 62e7c97 (62e7c97), committed 2026-07-18T02:22:57Z.
PR: #9348.
Model family: openai

Verdict: CHANGES-REQUESTED

  • [P2] aragora/swarm/quorum_evidence.py:1666 switches the Kimi quorum reviewer to moonshotai/kimi-k3 immediately. OpenRouter lists K3 as released July 16, 2026, so on July 20 it is still inside the repo’s documented 14-day soak rule for merge-authority evidence; the PR does not add catalog soak metadata or gate adoption. Source: https://openrouter.ai/moonshotai/kimi-k3-20260715
  • [P2] aragora/pdb/invoker_factory.py:87 and aragora/agents/api_agents/openrouter.py:807 make K3 the default live Kimi model, but billing/metering mirrors still lack moonshotai/kimi-k3. Those paths fall back to OpenRouter defaults $2/$8 instead of K3’s $3/$15, underbilling usage and weakening budget enforcement.

dogfood: yes

@scarmani

Copy link
Copy Markdown
Collaborator Author

Exact-head soak and catalog disposition

PR: #9348
Exact head: 62e7c976a0c0e5bbba82c5855eeb58db35cd6588
Current main: 28afef6f2cd1783b5d65ca64be877043cef9b1c3
Observed: 2026-07-20T16:14Z

Do not collect or apply new evidence on this head. The current OpenAI P2 dissent is substantive, and the prior Tier-4 settlement authorization at 84fb93c... is stale.

Live classification:

  • OpenRouter's public catalog currently exposes moonshotai/kimi-k3, created 2026-07-16T15:30:58Z, with prompt/completion prices of $3/$15 per million tokens and a 1,048,576-token context. The model exists; availability is not the blocker.
  • The repository's canonical model catalog and 14-day soak-aware routing landed on current main in feat(models): canonical model catalog — phase 1, enforcement without rewiring #9355/feat(routing): project catalog pricing into the router — first Phase 2 consumer #9364 after this branch's merge base. K3 is not yet cataloged there. Its earliest normal adoption date is 2026-07-30.
  • This branch directly changes merge-authority reviewer selection and several live defaults before that boundary, while billing and metering mirrors lack the K3 row. That combination can both violate soak policy and underbill usage.
  • A synthetic additive merge with current main is conflict-free, but it leaves the same 42-file broad default-switch diff. The preserved branch worktree is clean and exactly synchronized with the remote; no live process, steering, or active lease exists.

Optimal disposition

Park this PR until the K3 soak boundary rather than adding another ad hoc price table or settling current-head dissent.

On or after 2026-07-30, re-ground against the live OpenRouter catalog and current main, then perform one bounded catalog-first repair on this existing branch:

  1. merge current main additively without rewriting published history;
  2. add K3 once to aragora/models/catalog.py and its immutable snapshot with release/soak dates, context, and live $3/$15 pricing;
  3. let catalog enforcement drive every required billing, metering, PDB, selector, and routing mirror, with focused parity tests and zero conservative-default fallback;
  4. change defaults and the quorum reviewer only after ModelSpec.is_under_soak() is false;
  5. correct or remove the unsupported kimi-thinking "mandatory-reasoning" claim unless a distinct mode is actually wired;
  6. run fresh exact-head Claude/OpenAI review and obtain fresh Tier-4 settlement.

If K3's live ID, price, context, or availability changes before the boundary, keep the PR parked and update the catalog evidence rather than preserving stale values.

No branch, CI, evidence, settlement, merge, label, or protection state was changed by this packet.

scarmani added a commit that referenced this pull request Jul 24, 2026
…y record [Tier 4] (#9363)

* fix(swarm): remove gemini from the counting quorum set; kimi lane to K3

2026-07-16 founder directive following a repeat fabricated-claim pattern in
gemini merge-quorum reviews (invented model release dates asserted twice
after refutation, false METRICS-drift claims, demands for nonexistent route
ids — full record in operator-context 20260716T2200Z). Gemini reviews still
post and remain readable as advisory evidence; they no longer count toward
Tier 3-4 quorums or satisfy the Tier-2 Western condition.

The kimi advisory lane upgrades moonshotai/kimi-k2.6 -> kimi-k3
(live-verified 2026-07-16: $3/$15 per MTok, 1M context). Counting set
retains claude/openai/grok/mistral/hermes. Promoting kimi (moonshot) into
the counting set would amend the jurisdiction principles doc and is left as
an explicit founder decision, not folded in here.

TIER 4: quorum eligibility surface — requires founder exact-head settlement.

259+3 quorum-evidence tests green.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(governance): align roster pin test + authority principles doc with gemini demotion

Quorum round-1 findings at 7659c55:
- P1: test_western_families_match_spec now pins the post-directive roster
  (gemini removed); no other tests assert the old counting set (recognizer/
  retrigger gemini references are recognition-layer, intentionally kept).
- P2: REVIEW_AUTHORITY_PRINCIPLES.md drops Google from the Western counting
  list, documents the gemini advisory-only demotion citing the 2026-07-16
  reviewer-reliability record, and notes the demotion is itself a Tier-4
  change made with operator preapproval and settled through the Tier-4 chain.
- P3 (note): documented the ARAGORA_OPENROUTER_REVIEWER_MODELS JSON override
  for advisory-lane model pins (kimi-k3) and its contained failure mode.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(governance): narrow PR to gemini demotion; commit auditable evidence record

Quorum round-2 findings at 416fd85:
- P2 (claude): drop the kimi K3 lane flip — revert to moonshotai/kimi-k2.6.
  The reliability record itself defers K3 until live verification + a
  catalog entry exist (catalog work #9348 is unmerged). Test pin updated;
  doc notes the deferral explicitly.
- P2 (openai): the Tier-4 evidence artifact was gitignored. Committed
  verbatim as docs/governance/records/20260716T2200Z-gemini-reviewer-
  reliability-record.md with a provenance header; all citations
  (quorum_evidence.py, REVIEW_AUTHORITY_PRINCIPLES.md, governance test)
  repointed to the committed path, plus a test pinning its presence.
- P3 (openai): removed Google from the stale WESTERN_FAMILIES description
  comment.
- P3 (claude): doc now states Chinese-routed lanes fully count at Tier 0-2
  per the tier table (model-pin changes do not alter counting authority),
  and explicitly frames the reliability record + this PR's Tier-4
  settlement as an operator-approved substitute for the docs/specs/
  design-doc requirement, scoped to this demotion only.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(swarm): advisory-only is a first-class family classification — gemini never counts for or against, at any tier

Quorum round-3 findings at 975bda0 (both reviewers, same P2): dropping
gemini from WESTERN_FAMILIES only stopped its supportive signal at
Western-keyed tiers; it still counted at Tier 0-1 and its CHANGES-REQUESTED
still promoted blocking dissent. The record's mandate is 'gemini dissent is
NOT to be counted anywhere'.

- ADVISORY_ONLY_FAMILIES = {gemini} (cites the committed record), plus
  CHINESE_ROUTED_FAMILIES so the classification is explicit and TOTAL:
  every FAMILY_PROVIDERS key belongs to exactly one of western /
  chinese-routed / advisory-only (partition pinned by governance tests).
- TierQuorumRule.counted_families drops advisory-only families at EVERY
  tier — covers all three rule-derived surfaces (auto-settle, review-queue
  signal_count, reconcile diagnostic) even for raw reviewer-id lists.
- EvidenceItem: advisory-only would_count demoted at the shared __post_init__
  choke point (prepared artifacts cannot smuggle it back); .dissenting is
  False for advisory-only families.
- review_queue: _dissenting_views_from_comments never promotes advisory-only
  blocking dissent; _get_validated_review_classification skips advisory-only
  reviews before the blocking scan and heard/dissent accounting (an
  advisory-only P0/P1 cannot veto advisory_settle either). Reviews still
  post/parse/lint — advisory visibility preserved everywhere.
- [P3] taxonomy: doc + code comment now classify western / chinese-routed /
  advisory-only explicitly; payload-jurisdiction maps gemini explicitly to
  the Western (Google/US) column — demotion removes counting authority, not
  payload eligibility. Stale Google mention in the WESTERN_FAMILIES comment
  fixed.
- [P3] design-doc-waiver text moved out of REVIEW_AUTHORITY_PRINCIPLES.md
  into a clearly-marked in-repo addendum on the committed record; the
  principles doc keeps the roster fact + citation only.
- Tests: partition totality/disjointness; gemini PASS never counts (tiers
  0-4 x both gate regimes); gemini CHANGES-REQUESTED (P1-backed) never
  blocks while the same review from claude still does; outcome-level
  dissenting/counting exclusion.

Note: tests/swarm/test_pr_review_protocol.py::
test_resolve_provider_slots_prefers_available_candidates fails pre-existing
on this branch (fails with this diff stashed too); unrelated.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(swarm): close advisory-only enforcement gaps — dogfood leg, protocol dissent, packet visibility, alias ids

Quorum round-4 findings at d55528c (claude CR, openai PASS):
- P2: the required adversarial-dogfood leg no longer accepts an
  advisory-only-attributed item (a gemini dogfood comment could satisfy the
  Tier 1+ requirement via _known_model_reviewer_id). It stays visible in
  dogfood_evidence but does not satisfy the leg.
- P2: protocol-payload dissenting_views are now filtered for advisory-only
  identities like the comments path (a {agent: gemini} view from a
  merge-protocol payload or stale prepared artifact no longer blocks);
  shared helper _view_is_advisory_only checks model_family + agent
  (family:role form), canonicalized.
- P3: a blocked-severity ([P0]/[P1]) advisory-only CHANGES-REQUESTED no
  longer vanishes from the merge packet — recorded as an advisory view,
  with the reasons note naming the roster demotion (advisory-only family)
  rather than the severity gate as the non-blocking cause.
- P3: counted_families canonicalizes via canonical_family before the
  advisory-only subtraction, and _FAMILY_ALIASES gains google -> gemini
  (mirrors FAMILY_PROVIDERS / the recognizer markers), so raw
  alias/provider ids cannot dodge the exclusion at any filter site.
- Tests, one per gap: gemini dogfood fails the required leg (deepseek
  contrast satisfies); protocol-payload gemini dissent does not block;
  P1-backed gemini review appears in advisory_views with the
  advisory-only-family reason; alias forms (google/Google/' GEMINI ')
  excluded across tiers, EvidenceItem counting, and dissent.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* refactor(cli): extract review-queue renderers to review_queue_render.py

review_queue.py sits at its 6000-LOC bridge ceiling (#8553: extract, do
not grow) and this PR's advisory-only roster logic pushed it to 6054.
Move the pure presentation helpers (_render_packet,
_render_merge_authorization_packet, _render_active_auto_handle_alerts)
to a satellite module, re-exported under their historical names so call
sites and tests are unchanged. 5783 LOC; ratchet PASS.

* fix(quorum): advisory-only families excluded from emitted counted ids

_counted_model_reviewer_ids now drops advisory-only families at the
source, so merge packets (counted_reviewer_ids/counted_model_families)
and evidence-lint would_count match the never-count roster contract —
downstream automation reads those fields as counts-toward-quorum
(#9363 round-5 openai [P2]x2). Reviews stay visible as
reviewer_signals/advisory_views. Regression tests for both surfaces.

* fix(quorum): canonicalize gemini-cli registry id to the gemini family

Live protocol payloads carry AgentRegistry names ('gemini-cli:role');
without the alias, demoted gemini dissent re-enters through
protocol dissenting_views and dodges the advisory-only exclusion
(#9363 round-5 openai [P2]). Regression tests on both surfaces.

* docs(governance): align REVIEW_AUTHORITY_PRINCIPLES with the counted-family code

Tier 3-4 counts only the frontier-grade Western subset (claude, openai,
grok); the doc still said all Western families count at every tier, an
authoritative spec/code mismatch about merge authority (#9363 round-6
openai [P2]). Kimi lane example refreshed to the live catalogued pin and
the lifted K3 deferral replaced with the surviving requirement — a
reviewer pin must name a catalogued model ([P3]).

* fix(quorum): close the antigravity gemini-demotion escape

antigravity is the current primary Gemini surface (agy CLI,
default_model=gemini-3.5-flash) but its registry id did not canonicalize
to gemini, so a protocol dissenting_view attributed to
'antigravity:<role>' passed _view_is_advisory_only and BLOCKED merges —
exactly the 'gemini dissent counted somewhere' this PR exists to prevent
(#9363 round-6 claude [P2]).

Rather than hand-adding a third alias and waiting for the next surface,
test_every_gemini_registry_surface_is_demoted walks AgentRegistry and
fails CI on any Gemini-family agent whose id escapes the demotion.
Verified the guard catches the leak with the alias removed.

Also recognizes 'antigravity' as a gemini marker on both recognizer
tables so a self-labelled Antigravity review is preserved as an advisory
view instead of dropped ([P3]).

---------

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant