fix(plugin-openai): skip agent_config_update items in realtime chat-ctx sync

[tsushanth]

Summary

Fixes #1855 — when an Agent is constructed with tools or instructions, AgentActivity inserts internal agent_config_update chat items on enter and on tool / instructions changes (agents/src/voice/agent_activity.ts:520, :875). The OpenAI Realtime plugin's chat-context sync (createChatCtxUpdateEvents in plugins/openai/src/realtime/realtime_model.ts:662) feeds these into livekitItemToOpenAIItem, which has no agent_config_update case and throws:

Unsupported item type: agent_config_update
  at livekitItemToOpenAIItem (realtime_model.js)
  at RealtimeSession.createChatCtxUpdateEvents
  at RealtimeSession.updateChatCtx
  at AgentActivity._realtimeGenerationTaskImpl

The throw is caught upstream — non-fatal — but the never-played-message sync bails mid-iteration, leading to degraded post-tool speech behavior and missing transcript items on those turns (reporter's observation).

Fix

Filter agent_config_update items out of newChatCtx.items inside createChatCtxUpdateEvents before the diff is computed:

newChatCtx.items = newChatCtx.items.filter((item) => item.type !== 'agent_config_update');

This mirrors the existing patterns elsewhere in the framework:

• Non-realtime path — agent_activity.ts:666 already uses .copy({ excludeInstructions: true, excludeHandoff: true, excludeConfigUpdate: true }) when handing context to non-realtime LLMs.
• Distributed sessions wire — remote_session.ts:406 already filters item.type !== 'agent_config_update' before transport.

livekitItemToOpenAIItem's defensive throw is left in place — it's still the correct behavior for unknown item types reaching the converter through other code paths.

Tests

Two new cases in plugins/openai/src/realtime/realtime_model.test.ts:

• Mixed ChatContext (system + agent_config_update + user) — createChatCtxUpdateEvents emits conversation.item.create only for the system and user messages; the config update id is absent from the events.
• ChatContext containing only agent_config_update items — resolves to [] (no events, no throw).

Reverse-verified: both new tests fail RED with the exact reporter error against upstream/main, pass after the filter is added.

Local: pnpm vitest run plugins/openai/src/realtime/realtime_model.test.ts → 35/35 green. pnpm lint clean (36/36 packages). Pre-existing @livekit/local-inference / @livekit/agents-plugins-test import errors during pnpm build:agents are unrelated and reproduce on upstream/main with this change stashed.

Changeset: @livekit/agents-plugin-openai patch.

Breaking changes

None. agent_config_update items had no OpenAI Realtime representation to begin with — the only reachable behavior change is no longer throwing on them during sync.

[changeset-bot]

🦋 Changeset detected

Latest commit: 9957723

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 35 packages

Name	Type
@livekit/agents-plugin-openai	Patch
@livekit/agents-plugin-anam	Patch
@livekit/agents-plugin-cartesia	Patch
@livekit/agents-plugin-cerebras	Patch
@livekit/agents-plugin-elevenlabs	Patch
@livekit/agents-plugin-fishaudio	Patch
@livekit/agents-plugin-google	Patch
@livekit/agents-plugin-hume	Patch
@livekit/agents-plugin-inworld	Patch
@livekit/agents-plugin-neuphonic	Patch
@livekit/agents-plugin-perplexity	Patch
@livekit/agents-plugin-rime	Patch
@livekit/agents-plugin-sarvam	Patch
@livekit/agents-plugin-xai	Patch
@livekit/agents	Patch
@livekit/agents-plugin-assemblyai	Patch
@livekit/agents-plugin-baseten	Patch
@livekit/agents-plugin-bey	Patch
@livekit/agents-plugin-deepgram	Patch
@livekit/agents-plugin-did	Patch
@livekit/agents-plugin-hedra	Patch
@livekit/agents-plugin-lemonslice	Patch
@livekit/agents-plugin-liveavatar	Patch
@livekit/agents-plugin-livekit	Patch
@livekit/agents-plugin-minimax	Patch
@livekit/agents-plugin-mistral	Patch
@livekit/agents-plugin-mistralai	Patch
@livekit/agents-plugin-phonic	Patch
@livekit/agents-plugin-resemble	Patch
@livekit/agents-plugin-runway	Patch
@livekit/agents-plugin-silero	Patch
@livekit/agents-plugin-soniox	Patch
@livekit/agents-plugin-tavus	Patch
@livekit/agents-plugin-trugen	Patch
@livekit/agents-plugins-test	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[tsushanth]

Thanks @devin-ai-integration — good catch. agent_handoff is in the ChatItem union and livekitItemToOpenAIItem's default arm throws on it too. Even if it isn't reachable on the realtime path today, the non-realtime path's chatCtx.copy({ excludeHandoff: true, excludeConfigUpdate: true }) (agent_activity.ts:666) explicitly excludes both — same hazard, same precedent.

Filtered agent_handoff alongside agent_config_update in 9957723 and added a regression test that pushes an AgentHandoffItem next to a user message and asserts the user message still gets a create event while the handoff id is absent. Tests now 36/36 green.