fix: persist API keys when switching LLM providers

[shivangtanwar]

Summary

Fixes #674

When users switch between AI providers (e.g., OpenAI → Anthropic) in the Settings page, the previous provider's API key is lost. Switching back requires re-entering the key.

Root cause: The config stores a single flat api_key field that gets overwritten on every provider switch. The per-provider api_keys dictionary exists but was never used during the switch flow.

Changes

apps/backend/app/routers/config.py:

• PUT /config/llm-api-key — Before switching providers, saves the current api_key into api_keys[old_provider]. When switching to a new provider with no explicit key, restores from api_keys[new_provider] if available.
• GET /config/llm-api-key — Falls back to api_keys[provider] when the top-level api_key is empty, so the UI correctly shows a stored key exists.

How it works

config.json before switch (OpenAI → Anthropic):
{
  "provider": "openai",
  "api_key": "sk-openai-xxx",
  "api_keys": {}
}

config.json after switch:
{
  "provider": "anthropic",
  "api_key": "",              ← cleared (user hasn't entered Anthropic key yet)
  "api_keys": {
    "openai": "sk-openai-xxx" ← preserved!
  }
}

config.json after switching back to OpenAI:
{
  "provider": "openai",
  "api_key": "sk-openai-xxx", ← automatically restored
  "api_keys": {
    "openai": "sk-openai-xxx"
  }
}

Test plan

• Configure OpenAI with an API key, save, verify it works
• Switch to Anthropic — verify OpenAI key is NOT lost (check data/config.json)
• Enter Anthropic key, save, verify it works
• Switch back to OpenAI — verify the key is auto-restored (masked key shown in UI)
• Switch to Ollama (no key required) and back — verify keys still persist

---

Summary by cubic

Persist API keys when switching LLM providers so keys aren’t lost and are restored automatically. Fallback to per‑provider keys now powers both the UI and runtime LLM calls.

• Bug Fixes
  • Save the old provider’s key into api_keys before switching; if no new key is provided, restore the saved key for the target provider.
  • Fall back to per‑provider keys when the top‑level key is empty in both GET /config/llm-api-key and llm.get_llm_config, with a defensive type check for api_keys.

^{Written for commit cd48f8c. Summary will update on new commits.}

[kilo-code-bot]

Code Review Summary

Status: 3 Issues Found | Recommendation: Address before merge

Overview

Severity	Count
CRITICAL	1
WARNING	2
SUGGESTION	0

Issue Details (click to expand)

CRITICAL

File	Line	Issue
apps/backend/app/routers/config.py	98	Inconsistent API key fallback logic - already commented, author fixed in c5f3aee

WARNING

File	Line	Issue
apps/backend/app/routers/config.py	132	Missing fallback to settings.llm_api_key when saving old key - author responded env var keys are ephemeral
apps/backend/app/routers/config.py	147	Empty string vs None handling - author confirmed intentional behavior

Other Observations (not in diff)

Issues found in unchanged code that cannot receive inline comments:

File	Line	Issue
apps/backend/app/routers/config.py	196-199	WARNING: test_llm_connection endpoint does NOT use the per-provider API key fallback logic. When a user switches providers and the top-level api_key is empty, the GET endpoint will show the per-provider key, but the test endpoint will fail to find it. This creates an inconsistent user experience where the displayed key doesn't match what's tested. Consider applying the same fallback pattern here.

Files Reviewed (2 files)

• apps/backend/app/llm.py - 0 new issues (per-provider fallback logic looks correct)
• apps/backend/app/routers/config.py - 3 issues (all previously commented)

---

LiteLLM Configuration Review

The PR adds per-provider API key storage and retrieval, which is a useful feature for users who switch between providers. The implementation:

1. get_llm_config() in llm.py - Correctly falls back to per-provider keys when top-level key is empty
2. GET /config/llm-api-key - Correctly uses per-provider fallback
3. PUT /config/llm-api-key - Saves old provider's key before switching and restores saved key for new provider

Prompts Flow: The LLM configuration is used by complete() and check_llm_health() functions which are called by the prompts/enrichment/refinement services. The per-provider fallback ensures the correct API key is used regardless of which provider is active.

---

Fix these issues in Kilo Cloud

[cubic-dev-ai]

1 issue found across 1 file

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="apps/backend/app/routers/config.py">

<violation number="1" location="apps/backend/app/routers/config.py:135">
P2: Switching providers can overwrite a newer per-provider key with a stale top-level api_key because the new preservation block always writes stored["api_key"] into api_keys[old_provider], while /config/api-keys only updates api_keys. This creates a real regression when both endpoints are used.</violation>
</file>

_{Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.}

[cubic-dev-ai]

3 issues found across 2 files (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="apps/backend/app/routers/config.py">

<violation number="1" location="apps/backend/app/routers/config.py:137">
P2: Guarding the per-provider key update can leave stale keys in api_keys and discard recent /llm-api-key updates when switching providers.</violation>
</file>

<file name="apps/backend/app/llm.py">

<violation number="1" location="apps/backend/app/llm.py:252">
P2: Explicit empty api_key is no longer respected; falsy check now falls back to per-provider/env defaults, masking missing-key state after provider switches and potentially using unrelated env credentials.</violation>

<violation number="2" location="apps/backend/app/llm.py:253">
P2: `stored.get("api_keys", {})` can return a non-dict persisted value (e.g., null/string), causing `.get(...)` to raise AttributeError and break config loading.</violation>
</file>

_{Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.}

[cubic-dev-ai]

1 issue found across 2 files (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="apps/backend/app/routers/config.py">

<violation number="1" location="apps/backend/app/routers/config.py:135">
P1: Provider switching now unconditionally overwrites per-provider API keys, which can clobber newer keys saved via `/config/api-keys`.</violation>
</file>

_{Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.}