Skip to content

feat(ai-gateway): add organization auto model routing#4056

Open
pandemicsyn wants to merge 10 commits into
mainfrom
florian/feat/org-defined-auto
Open

feat(ai-gateway): add organization auto model routing#4056
pandemicsyn wants to merge 10 commits into
mainfrom
florian/feat/org-defined-auto

Conversation

@pandemicsyn

@pandemicsyn pandemicsyn commented Jun 17, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Adds kilo-auto/org, an Enterprise-only Organization Auto model that lets an organization choose which model should handle each Kilo mode.
  • Organization Auto stores routes in organizations.settings.org_auto_model:
    • If a request mode has an explicit route, Organization Auto uses that model.
    • If a request mode has no explicit route, Organization Auto uses the configured fallback model.
    • If a request uses an unknown/custom mode, Organization Auto also uses the fallback model unless that exact mode slug has a route.
    • Exact route slugs win first, so build uses routes.build before falling back to routes.code, and plan uses routes.plan before falling back to routes.architect.
  • Replaces the unshipped org-default-model-config mode-default rollout with the new organization-auto-model-routing PostHog toggle for canonical organization-wide routing.
  • Adds Enterprise-only enable/disable/fallback controls, canonical mode route configuration, route-aware mode lifecycle behavior, and exact route audit messages.
  • Replaces the default-model dialog with one unified behavior save flow for Organization Auto, a specific model, or the global default.
  • Organization Auto is presented as the recommended path with accessible radio semantics, keyboard navigation, and explicit selected-state affordances.
  • Reset to global default now appears even when Organization Auto is active.
  • Reset now explicitly disables Organization Auto and returns the organization to global-default resolution while preserving dormant routes and fallback settings for future re-enable.
  • Custom Modes now labels stored routes as inactive when Organization Auto is off instead of implying they are currently applied.
  • Removes the unshipped OrganizationModeConfig.defaultModel bridge and stores all routing only in organizations.settings.org_auto_model.
  • Adds organization-owned direct BYOK route support with credential ownership checks, direct-provider routing, zero Kilo hosted cost, and no fallback to OpenRouter/Vercel when credentials disappear.
  • Adds client-safe shared Organization Auto helpers so browser components do not import server-only DB/pg modules.

Verification

  • Verified the default behavior chooser in the browser, including selected-state styling, the Recommended badge, and focused-but-unselected options.
  • Verified keyboard navigation for the radio chooser with ArrowUp/Down/Left/Right, Home, and End.
  • Verified Organization Auto and Specific model dialog states at the compact dialog width.
  • Verified Organization Auto API behavior manually with /tmp/org-auto-api-log.sh against a local service, including modes/defaults/models and optional gateway requests.

Visual Changes

Screenshot 2026-06-17 at 5 01 46 PM Screenshot 2026-06-17 at 5 02 18 PM Screenshot 2026-06-17 at 5 02 31 PM Screenshot 2026-06-17 at 5 03 28 PM

Reviewer Notes

  • The feature is intentionally scoped to canonical org_auto_model settings, gateway resolution, BYOK ownership checks, route lifecycle consistency, and UI configuration.
  • Organization Auto UI/configuration writes are gated by the PostHog release toggle organization-auto-model-routing, replacing the old org-default-model-config toggle. Runtime routing remains data-driven for already-configured organizations, and disable remains available during rollback.
  • Stripe webhook/admin plan transitions intentionally do not clear the stored sentinel; catalog/default/runtime eligibility gates make dormant routes safe while preserving future re-enable behavior. The explicit global reset path now disables active Organization Auto while preserving those dormant route settings.
  • organization-auto-model-shared.ts exists specifically to keep client components from importing server-only DB/pg code.
  • Review route audit messages, BYOK fallback protection, and route-management authorization as the highest-risk areas.

Comment thread apps/web/src/lib/organizations/organization-auto-model.ts Outdated
@kilo-code-bot

kilo-code-bot Bot commented Jun 17, 2026

Copy link
Copy Markdown
Contributor

Code Review Summary

Status: No Issues Found | Recommendation: Merge

Executive Summary

Single commit addresses the previously flagged audit log issue by capturing previousDefaultModel inside the mutation callback before the organization settings are modified. A new test assertion verifies the correct audit message for the Organization Auto → global transition. No new issues found.

Files Reviewed (2 incremental, 42 total)
  • apps/web/src/routers/organizations/organization-settings-router.ts - Fixed audit log (previous concern resolved)
  • apps/web/src/routers/organizations/organization-settings-router.test.ts - Added audit log assertion
Previous Review Summaries (5 snapshots, latest commit cf3f457)

Current summary above is authoritative. Previous snapshots are kept for context only.

Previous review (commit cf3f457)

Status: No Issues Found | Recommendation: Merge

Executive Summary

Refactors default model mutations to use mutateOrganizationSettings with for update locking, adds Organization Auto routing with BYOK support, and introduces a unified configureOrganizationDefaultBehavior endpoint. Previous audit log concern at line 878 resolved by the refactor — the pre-transaction existingOrg is now used correctly to determine audit message type. No new issues found.

Files Reviewed (42 files)
  • apps/web/src/app/(app)/organizations/[id]/custom-modes/page.tsx
  • apps/web/src/app/api/openrouter/[...path]/route.test.ts
  • apps/web/src/app/api/openrouter/[...path]/route.ts
  • apps/web/src/app/api/organizations/[id]/defaults/route.test.ts
  • apps/web/src/app/api/organizations/[id]/defaults/route.ts
  • apps/web/src/app/api/organizations/[id]/modes/route.test.ts
  • apps/web/src/app/api/organizations/[id]/modes/route.ts
  • apps/web/src/app/api/organizations/hooks.ts
  • apps/web/src/components/models/CondensedProviderAndModelsList.tsx
  • apps/web/src/components/organizations/OrganizationDashboard.tsx
  • apps/web/src/components/organizations/OrganizationProvidersAndModelsConfigurationCard.tsx
  • apps/web/src/components/organizations/custom-modes/CustomModesLayout.tsx
  • apps/web/src/components/organizations/custom-modes/EditModeForm.test.ts
  • apps/web/src/components/organizations/custom-modes/EditModeForm.tsx
  • apps/web/src/components/organizations/custom-modes/ModeForm.tsx
  • apps/web/src/components/organizations/custom-modes/NewModeForm.tsx
  • apps/web/src/components/organizations/providers-and-models/DefaultModelDialog.tsx
  • apps/web/src/components/organizations/providers-and-models/ModelsTab.tsx
  • apps/web/src/components/organizations/providers-and-models/OrganizationProvidersAndModelsPage.tsx
  • apps/web/src/components/organizations/providers-and-models/ProvidersTab.tsx
  • apps/web/src/lib/ai-gateway/auto-model/index.ts
  • apps/web/src/lib/ai-gateway/auto-model/resolution.test.ts
  • apps/web/src/lib/ai-gateway/auto-model/resolution.ts
  • apps/web/src/lib/ai-gateway/byok/index.ts
  • apps/web/src/lib/ai-gateway/llm-proxy-helpers.ts
  • apps/web/src/lib/ai-gateway/providers/openrouter/index.ts
  • apps/web/src/lib/organizations/organization-auto-model-shared.ts
  • apps/web/src/lib/organizations/organization-auto-model.ts
  • apps/web/src/lib/organizations/organization-base-types.ts
  • apps/web/src/lib/organizations/organization-models.ts
  • apps/web/src/lib/organizations/organization-modes.test.ts
  • apps/web/src/lib/organizations/organization-modes.ts
  • apps/web/src/lib/organizations/organization-seats.ts
  • apps/web/src/lib/organizations/organization-types.ts
  • apps/web/src/lib/organizations/organizations.ts
  • apps/web/src/lib/proxy-error-types.ts
  • apps/web/src/routers/organizations/organization-modes-router.test.ts
  • apps/web/src/routers/organizations/organization-modes-router.ts
  • apps/web/src/routers/organizations/organization-settings-router.test.ts
  • apps/web/src/routers/organizations/organization-settings-router.ts
  • packages/db/src/schema-types.test.ts
  • packages/db/src/schema-types.ts

Previous review (commit 29a354e)

Status: 1 Issue Found | Recommendation: Approve

Executive Summary

Refactors the default model dialog and custom modes UI with a unified behavior endpoint (auto/specific/global). One minor suggestion: the audit log inside the transaction uses pre-transaction state for the global-reset case.

Overview

Severity Count
CRITICAL 0
WARNING 0
SUGGESTION 1
Issue Details (click to expand)

SUGGESTION

File Line Issue
apps/web/src/routers/organizations/organization-settings-router.ts 878 Audit log uses pre-transaction existingOrg instead of fresh organization parameter
Files Reviewed (9 files)
  • apps/web/src/app/api/organizations/hooks.ts
  • apps/web/src/components/models/CondensedProviderAndModelsList.tsx
  • apps/web/src/components/organizations/custom-modes/CustomModesLayout.tsx
  • apps/web/src/components/organizations/custom-modes/EditModeForm.tsx
  • apps/web/src/components/organizations/custom-modes/ModeForm.tsx
  • apps/web/src/components/organizations/custom-modes/NewModeForm.tsx
  • apps/web/src/components/organizations/providers-and-models/DefaultModelDialog.tsx
  • apps/web/src/routers/organizations/organization-settings-router.test.ts
  • apps/web/src/routers/organizations/organization-settings-router.ts - 1 issue

Fix these issues in Kilo Cloud

Previous review (commit 412123c)

Status: No Issues Found | Recommendation: Merge

Executive Summary

Incremental changes cover a major tier-to-taxonomy refactoring across auto-routing contracts, decision engine, benchmarks, and admin UI, plus KiloClaw billing recovery UX, model catalog refresh for vision capabilities, cloud agent fork review flow, and CI/CD staging deployment. All changes are well-tested with no security, logic, or performance issues detected.

Files Reviewed (40+ files changed since last review)
  • packages/auto-routing-contracts/src/ — Tier → taxonomy refactoring (tiers.ts → taxonomy.ts + reasoning.ts)
  • services/auto-routing/src/ — Decision engine, tests, and index updated for taxonomy routes
  • services/auto-routing-benchmark/src/ — Decider sharding, org context, chunk pipeline rework
  • services/auto-routing-benchmark/**/migrations/ — D1 column renames (tier → route_key), benchmark_org_id
  • services/auto-routing-benchmark/container/ — kiloApiUrl + orgId support in CLI container
  • services/auto-routing-benchmark/wrangler.jsonc — max_instances 100, max_concurrency 100, retry tuning
  • services/kiloclaw/controller/src/model-catalog-refresh.ts — New periodic model catalog seed for vision
  • services/kiloclaw/controller/src/index.ts — Wiring for catalog refresh
  • apps/web/src/lib/ai-gateway/ — Efficient model now public, data collection check refactored, mirror removed
  • apps/web/src/lib/ai-gateway/auto-routing-decision.ts — buildDecidePayload inlined from deleted mirror module
  • apps/web/src/lib/ai-gateway/is-free-model.ts — hasBestEffortGuessDataCollectionRequirement
  • apps/web/src/lib/ai-gateway/providers/ — MiniMax m3 reasoning fix, GLM model variant fix, efficient model listing
  • apps/web/src/app/admin/auto-routing/ — Admin UI for taxonomy routes, benchmarkOrgId, costPerAccuracy
  • apps/web/src/app/api/internal/auto-routing-benchmark/token/ — Org-scoped token minting
  • apps/web/src/app/cloud-agent-fork/review/ — New fix-review redirect flow with prompt-based sessions
  • apps/web/src/lib/code-reviews/prompts/fix-review-prompt.ts — New prompt builder for review fix workflow
  • apps/web/src/lib/code-reviews/review-memory/github-feedback.ts — Repository permission gating for feedback recording
  • apps/web/src/lib/integrations/platforms/github/adapter.ts — Exported CollaboratorPermission type
  • apps/web/src/lib/kiloclaw/credit-billing.ts — CreditEnrollmentError with typed reasons, instance/price version gating
  • apps/web/src/lib/kiloclaw/pricing-catalog.test.ts — resolveKiloClawEnrollmentPriceVersion tests
  • apps/web/src/lib/kiloclaw/stripe-handlers.ts — Conditional logging on subscription.created processing
  • apps/web/src/app/(app)/claw/components/billing/ — Funding choice copy, hosting recovery UX
  • apps/web/src/app/payments/kilo-pass/awarding/ — Recovery UI with per-reason copy and retry
  • apps/web/src/routers/kilo-pass-router.ts — Structured activateCheckoutHosting with retryable/action_required outcomes
  • apps/web/src/routers/kiloclaw-router.ts — Enrollment error classification, destroyed/canceled gating
  • apps/web/src/routers/kiloclaw-billing-router.test.ts — Tests for checkout rejections and webhook ordering
  • packages/db/src/kiloclaw-pricing-catalog.ts — resolveKiloClawEnrollmentPriceVersion helper
  • .github/workflows/ — Staging deployment workflow, target_environment parameterization
  • apps/web/src/app/(app)/claw/components/changelog-data.ts — Vision fix changelog entry

Previous review (commit b3107a9)

Status: No Issues Found | Recommendation: Merge

Executive Summary

The previous WARNING about getBYOKforOrganization reading outside transaction isolation has been fixed. validateOrganizationAutoTarget now accepts an optional dbClient and all transaction-bound callers correctly pass tx. No new issues detected in the incremental changes.

Files Reviewed (4 files changed since last review)
  • apps/web/src/lib/ai-gateway/byok/index.ts
  • apps/web/src/lib/organizations/organization-auto-model.ts
  • apps/web/src/routers/organizations/organization-modes-router.ts
  • apps/web/src/routers/organizations/organization-settings-router.ts

Previous review (commit 6e0261a)

Status: 1 Issue Found | Recommendation: Address before merge

Executive Summary

validateOrganizationAutoTarget reads BYOK credentials outside the mutateOrganizationSettings transaction isolation context, creating a narrow consistency gap during Organization Auto route configuration.

Overview

Severity Count
CRITICAL 0
WARNING 1
SUGGESTION 0
Issue Details (click to expand)

WARNING

File Line Issue
apps/web/src/lib/organizations/organization-auto-model.ts 85 getBYOKforOrganization(db, ...) uses global db inside transaction callbacks, bypassing isolation
Files Reviewed (41 files)
  • apps/web/src/app/(app)/organizations/[id]/custom-modes/page.tsx
  • apps/web/src/app/api/openrouter/[...path]/route.test.ts
  • apps/web/src/app/api/openrouter/[...path]/route.ts
  • apps/web/src/app/api/organizations/[id]/defaults/route.test.ts
  • apps/web/src/app/api/organizations/[id]/defaults/route.ts
  • apps/web/src/app/api/organizations/[id]/modes/route.test.ts
  • apps/web/src/app/api/organizations/[id]/modes/route.ts
  • apps/web/src/app/api/organizations/hooks.ts
  • apps/web/src/components/models/CondensedProviderAndModelsList.tsx
  • apps/web/src/components/organizations/OrganizationDashboard.tsx
  • apps/web/src/components/organizations/OrganizationProvidersAndModelsConfigurationCard.tsx
  • apps/web/src/components/organizations/custom-modes/CustomModesLayout.tsx
  • apps/web/src/components/organizations/custom-modes/EditModeForm.test.ts
  • apps/web/src/components/organizations/custom-modes/EditModeForm.tsx
  • apps/web/src/components/organizations/custom-modes/ModeForm.tsx
  • apps/web/src/components/organizations/custom-modes/NewModeForm.tsx
  • apps/web/src/components/organizations/providers-and-models/DefaultModelDialog.tsx
  • apps/web/src/components/organizations/providers-and-models/ModelsTab.tsx
  • apps/web/src/components/organizations/providers-and-models/OrganizationProvidersAndModelsPage.tsx
  • apps/web/src/components/organizations/providers-and-models/ProvidersTab.tsx
  • apps/web/src/lib/ai-gateway/auto-model/index.ts
  • apps/web/src/lib/ai-gateway/auto-model/resolution.test.ts
  • apps/web/src/lib/ai-gateway/auto-model/resolution.ts
  • apps/web/src/lib/ai-gateway/llm-proxy-helpers.ts
  • apps/web/src/lib/ai-gateway/providers/openrouter/index.ts
  • apps/web/src/lib/organizations/organization-auto-model-shared.ts
  • apps/web/src/lib/organizations/organization-auto-model.ts - 1 issue
  • apps/web/src/lib/organizations/organization-base-types.ts
  • apps/web/src/lib/organizations/organization-models.ts
  • apps/web/src/lib/organizations/organization-modes.test.ts
  • apps/web/src/lib/organizations/organization-modes.ts
  • apps/web/src/lib/organizations/organization-seats.ts
  • apps/web/src/lib/organizations/organization-types.ts
  • apps/web/src/lib/organizations/organizations.ts
  • apps/web/src/lib/proxy-error-types.ts
  • apps/web/src/routers/organizations/organization-modes-router.test.ts
  • apps/web/src/routers/organizations/organization-modes-router.ts
  • apps/web/src/routers/organizations/organization-settings-router.test.ts
  • apps/web/src/routers/organizations/organization-settings-router.ts
  • packages/db/src/schema-types.test.ts
  • packages/db/src/schema-types.ts

Fix these issues in Kilo Cloud


Reviewed by deepseek-v4-pro-20260423 · 162,717 tokens

Review guidance: REVIEW.md from base branch main

@pandemicsyn

Copy link
Copy Markdown
Contributor Author

Fixed review feedback in b3107a9: BYOK ownership validation now accepts and uses the active transaction client inside Organization Auto settings/mode mutations.

Comment thread apps/web/src/routers/organizations/organization-settings-router.ts Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant