Repo: livekit/agents (Python) · observed on livekit-agents==1.5.2
Type: bug / UX gap · Area: voice (AgentSession pipeline), llm, error surfacing
Suggested labels: bug, enhancement, voice, dx

Summary

When the LLM endpoint returns HTTP 429 with body {"type": "inference_quota_exceeded", ...} (the project is out of LiveKit Inference credits / hit a quota), a voice agent joins the room, publishes its audio track, and then never speaks. There is no spoken message, nothing rendered in the Agent Builder preview, and — for the first few turns — not even a session close. From the caller's perspective the agent is simply, silently dead.

The gateway's behavior here is correct: it returns a well-formed, fully structured 429 (status code, type, human-readable hint, quota_type, category, remaining_limit, documentation_url). The gap is entirely on the SDK surfacing side — the SDK emits an error event but, by default, turns an unrecoverable LLM failure into dead air rather than anything a human (or a frontend) can perceive.

Reported by a user in the LiveKit community forum (LLM out of credits → "agent joins but never speaks", confirmed by staff as depleted inference credits, not a connection/config problem).

Reproduction

1. Build a standard pipeline agent: AgentSession(stt=deepgram, llm=<OpenAI-compatible / LiveKit Inference, e.g. gpt-5.2-chat>, tts=elevenlabs).
2. Use a project whose LLM token credit quota is exhausted (or otherwise force the LLM endpoint to return 429 inference_quota_exceeded).
3. Join and speak to the agent.

Expected: the caller gets some perceptible signal — a spoken "this assistant is temporarily unavailable" message, and/or a frontend-renderable error/close state ("out of credits").

Actual: STT transcribes fine, then silence. The agent track is published but emits no audio. No reply, no spoken error. The Agent Builder preview shows nothing. The session does not even close until several consecutive failures (see root cause).

The upstream response the SDK receives

HTTP 429, JSON body (field names are stable; this is the LiveKit Inference gateway contract):

{
  "type": "inference_quota_exceeded",
  "error": "LLM token credit quota exceeded, category: MaxGatewayCredits, remaining_limit: 0",
  "hint": "LLM token credit quota exhausted. Wait for the next billing cycle or upgrade your plan.",
  "quota_type": "llm",
  "category": "MaxGatewayCredits",
  "current_usage": "...",
  "remaining_limit": "0",
  "status": "...",
  "free_tier": "false",
  "documentation_url": "https://livekit.com/pricing"
}

Everything a good UX needs (type, hint, quota_type) is right there in the body. The SDK just never looks at it.

Root cause (traced in livekit-agents==1.5.2)

1. A 429 is classified non-retryable, which is correct for quota — but means it raises straight through.
APIStatusError marks every 4xx (incl. 429) as retryable=False:

# livekit/agents/_exceptions.py:66-70
if retryable is None:
    retryable = True
    if status_code >= 400 and status_code < 500:   # 429 falls here
        retryable = False

For inference_quota_exceeded that classification is desirable — retrying won't help until the billing window resets. The decoded JSON body is preserved on APIError.body (_exceptions.py:23-29), so the rich quota detail is technically available to callers.

2. LLMStream emits an error event and re-raises — no audible side effect.

# livekit/agents/llm/llm.py:227-229
if self._conn_options.max_retry == 0 or not e.retryable:
    self._emit_error(e, recoverable=False)   # -> self._llm.emit("error", LLMError(...))
    raise

3. The error propagates to AgentSession as an ErrorEvent…

# livekit/agents/voice/agent_activity.py:1322-1324, 1345
if isinstance(error, llm.LLMError):
    error_event = ErrorEvent(error=error, source=self.llm)
    self._session.emit("error", error_event)
...
self._session._on_error(error)

ErrorEvent / CloseEvent / CloseReason already exist (voice/events.py:222-245). So a developer who explicitly registered @session.on("error") can see it — but nothing happens by default.

4. …and AgentSession._on_error deliberately does nothing for the first 3 unrecoverable LLM errors. This is the silence:

# livekit/agents/voice/agent_session.py:1363-1401  (max_unrecoverable_errors default = 3, line 129)
def _on_error(self, error: ...) -> None:
    if self._closing_task or error.recoverable:
        return
    if error.type == "llm_error":
        self._llm_error_counts += 1
        if self._llm_error_counts <= self.conn_options.max_unrecoverable_errors:
            return                      # <-- swallows the 1st, 2nd, 3rd failure entirely
    ...
    self._closing_task = asyncio.create_task(
        self._aclose_impl(error=error, reason=CloseReason.ERROR)   # only on the 4th
    )

The 3-strike tolerance is sensible for transient blips, but a 429 inference_quota_exceeded is terminal — it will fail identically every turn. So the agent absorbs three silent dead turns and then closes with CloseReason.ERROR and still no spoken explanation.

5. The reply turn produces no speech. The pipeline reply path awaits the inference tasks with wait_if_not_interrupted([*tasks]) (gathers with return_exceptions=True, agent_activity.py:2410). The raised 429 yields no reply text, so TTS is never invoked → the published audio track stays silent. (Unlike _tts_task_impl, the LLM reply path doesn't convert that failure into any fallback output.)

Net: error event fires (invisible unless subscribed) → no reply text → no TTS → silent track → repeats → session closes after 3 with no spoken reason. Hence "agent joins but never speaks."

Realtime path (RealtimeModel, e.g. Gemini Live / OpenAI Realtime) routes through the same _on_error → ErrorEvent mechanism (agent_activity.py:1325-1327), so an error-event-driven fix covers both pipeline and realtime.

Why this matters

• Silent failure is the worst failure mode for voice. A muted agent reads as "broken product," not "out of credits." The user has no way to self-diagnose.
• The diagnosis exists and is being thrown away. The gateway hands the SDK type: inference_quota_exceeded + a ready-to-speak hint. We surface none of it.
• First-party surfaces are affected too. The forum report notes the Agent Builder preview also showed nothing — so even LiveKit's own UI doesn't render this state.
• It's not just credits. The same path swallows any non-retryable LLM error (quota_type also covers stt/tts/bargein, and category includes rate-limit variants like MaxConcurrentGatewayLLMRpm/Tpm).

Proposed fix (layered — ship the cheap parts first)

A. Typed detection of quota/limit errors (core).
Give developers (and the SDK) a non-opaque way to recognize this. Either:

• introduce APIQuotaExceededError(APIStatusError), constructed when the response body has type == "inference_quota_exceeded" (carrying quota_type, category, hint, remaining_limit); or
• at minimum, guarantee the OpenAI/inference plugin populates APIError.body with the decoded JSON and document the field, so ev.error.body["type"] / ["hint"] are reliable.

B. Default user-perceptible surfacing (opt-in, sensible default).
On an unrecoverable LLM error with audio enabled, optionally speak a fallback line before closing — e.g. an AgentSession option on_unrecoverable_error (callback) or error_message: str | None. For the quota case, the gateway's hint is already caller-appropriate. Keep backwards compatible (default could be a generic "this assistant is temporarily unavailable").

C. Forward a structured signal to the room/frontend.
So frontends (incl. Agent Builder preview) can render an "out of credits / quota exceeded" state instead of dead air — e.g. surface the error/close events over the wire or publish a data/attribute event carrying type, quota_type, hint. This is what the forum reporter actually needed.

D. Don't silently absorb a terminal error 3×.
max_unrecoverable_errors=3 is right for transient errors but wrong for known-terminal ones. For inference_quota_exceeded (and other non-retryable quota categories), surface/close on the first occurrence rather than after three silent dead turns.

E. Docs + example (zero-risk, do immediately).
Document the inference_quota_exceeded body shape and the @session.on("error") + session.say(...) recipe; add an example. Extend the existing 429 test (tests/test_interruption/test_interruption_failover.py::test_retries_then_emits_unrecoverable) to assert the new surfaced/spoken path.

Acceptance criteria

• 429 inference_quota_exceeded is detectable via a typed error or documented body field — not an opaque blob.
• By default, or via a one-line documented option, the end user gets a perceptible signal (spoken message and/or frontend-renderable event) — never pure silence.
• A terminal quota error surfaces on the first occurrence, not after max_unrecoverable_errors.
• Agent Builder preview renders an "out of credits / quota exceeded" state for this case.
• Docs include the recipe + body contract; an example demonstrates it; a test covers 429-quota → surfaced.

Workaround for users today

from livekit.agents import ErrorEvent

@session.on("error")
def _on_error(ev: ErrorEvent):
    err = ev.error
    if getattr(err, "recoverable", True):
        return
    body = getattr(err, "body", None) or {}
    is_quota = (
        getattr(err, "status_code", None) == 429
        or (isinstance(body, dict) and body.get("type") == "inference_quota_exceeded")
    )
    if is_quota:
        msg = (body.get("hint") if isinstance(body, dict) else None) \
            or "Sorry — this assistant is temporarily unavailable due to usage limits. Please try again later."
        session.say(msg)
        # optionally: publish a data/attribute event so the frontend can render an "out of credits" state

References

• LiveKit community forum: LLM out-of-credits → "agent joins but never speaks" (staff-confirmed depleted inference credits): https://community.livekit.io/t/1282
• Gateway 429 contract (for maintainer reference): structured body built in agent-gateway pkg/quota/response.go (QuotaExceededResponse) / pkg/http/error.go (JSONErrorResponse); returned from pkg/handler/completions.go on QuotaStatusExceeded.
• SDK code paths (v1.5.2): _exceptions.py:48-96, llm/llm.py:211-259, voice/events.py:222-245, voice/agent_activity.py:1314-1345 & :2410, voice/agent_session.py:129 & :1363-1401.