Skip to content

Commit 4c00cb4

Browse files
committed
docs(learnings/voicemail-detection): add platform-VMD-vs-squad-handoff race fingerprint
When the gatekeeper assistant in a two-agent voicemail relay has platform `voicemailDetection: { provider: vapi }` configured AND no `voicemailMessage` field, the call silently breaks on every voicemail — bot never speaks, message never lands, ends with `endedReason: voicemail` after ~10–22s of dead air. Mechanism, verified via Axiom call timeline 019e2827 on 2026-05-14: - Carrier prompt streaming transcripts abort the gatekeeper LLM on every partial transcript (`model.requestAborted` storm). LLM never completes. - Platform VMD operates on raw audio in parallel, doesn't wait for LLM. Wins the race and ends the call. - Without `voicemailMessage` on the gatekeeper, the platform's "voicemail detected" outcome is a silent call end. No callback message plays. Fix: disable platform VMD on the gatekeeper. `assistantOverrides.voicemailDetection: null` in a squad-member override does NOT work — Vapi's API silently drops the field (verified via direct PATCH test on 2026-05-14). Either omit the field from the underlying assistant entirely, or fork the assistant without the field.
1 parent ba304b8 commit 4c00cb4

1 file changed

Lines changed: 28 additions & 0 deletions

File tree

docs/learnings/voicemail-detection.md

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -49,6 +49,34 @@ Outbound Call
4949
- The fronter prompt is clean — optimized solely for conversation quality
5050
- Each concern is independently testable and tunable
5151

52+
### ⚠️ Platform `voicemailDetection` MUST be disabled on the gatekeeper (Two-Agent Relay)
53+
54+
This is the single most non-obvious failure mode in two-agent voicemail relay setups. If the gatekeeper assistant has Vapi's platform `voicemailDetection: { provider: vapi }` configured, the call will **silently break on every voicemail** — the bot never speaks, the message never lands, the call ends with `endedReason: voicemail` after ~10–22s of dead air. We hit this on mudflap-test's iform voicemail triage squad on 2026-05-14 and only diagnosed it after pulling full Axiom event timelines.
55+
56+
**The mechanism (verified via call 019e2827 on 2026-05-14):**
57+
58+
1. The carrier prompt audio streams into the gatekeeper's transcriber (e.g. Soniox stt-rt-v4). Soniox emits a flood of `assistant.transcriber.partialTranscript` events.
59+
2. Each partial transcript triggers a new `pipeline.turnStarted`, which calls `assistant.model.clearing` and emits `assistant.model.requestAborted` on any in-flight LLM request. The gatekeeper LLM never gets a stable 1–2 second window to complete a response.
60+
3. **Meanwhile**, platform `voicemailDetection` runs the Vapi VMD classifier on the raw audio in parallel — it doesn't wait for the LLM. With `backoffPlan: { frequencySeconds: 2.5, maxRetries: 6, startAtSeconds: 2 }`, it fires `call.voicemailDetected` events starting at ~+5s into the call.
61+
4. Platform VMD ends the call with `endedReason: voicemail` regardless of whether the LLM has emitted its handoff tool call. The gatekeeper LLM literally never finishes a `model.firstTokenReceived` event — every request gets aborted by streaming transcripts before completion.
62+
5. Worse: if the gatekeeper has NO `voicemailMessage` field configured, platform VMD's "voicemail detected" outcome is to end the call **silently**. No callback message plays. The recording captures the carrier prompt + dead air.
63+
64+
**Architectural mismatch root cause:** Platform `voicemailDetection` + `voicemailMessage` is designed for the **single-agent** voicemail flow (platform detects → platform plays voicemailMessage → platform ends call). The two-agent relay's `handoff_to_voicemail_leaver` tool flow is a **competing detection path** that loses the race because:
65+
66+
- LLM-based detection requires the LLM to actually run to completion. Streaming transcripts during the carrier prompt prevent that.
67+
- Platform VMD operates directly on audio, in parallel, and doesn't need the LLM.
68+
- Without `voicemailMessage` on the gatekeeper, platform VMD's win = silent call end.
69+
70+
**Fix:** Disable platform `voicemailDetection` on the gatekeeper assistant. Either:
71+
72+
- **Recommended**: don't set the field at all on the gatekeeper. Multilingual triage classifiers that never had `voicemailDetection` configured (e.g. `iform-triage-classifier-multilingual-d98136d9`, `iform-triage-multilingual-classic-f6b53e27` on mudflap-test) work where same-shape squads with VMD-on classifiers failed.
73+
- **If you can't modify the underlying assistant** (e.g. a customer is gatekeeping the base classifier UUID for another reason): fork the classifier and use the fork in the squad. `assistantOverrides.voicemailDetection: null` does **NOT** work — Vapi's API silently drops the field. Verified via direct PATCH test on 2026-05-14.
74+
- **Never** set `voicemailDetection` on the gatekeeper AND rely on the handoff path. They are mutually exclusive architectures.
75+
76+
**Detection fallback when platform VMD is off:** The gatekeeper LLM's prompt is now the sole detector. Even with streaming-transcript aborts during the carrier prompt, the LLM eventually gets a stable window (the silence between carrier-prompt sentences, or after the prompt ends but before the beep) and emits its `handoff_to_voicemail_leaver` tool call. The leaver then speaks the callback message via the handoff's `request-start`.
77+
78+
**Engine-gap note for gitops users:** the `assistantOverrides.voicemailDetection: null` rejection is one of several Vapi-side override fields that silently drop a null in a squad-member override. If you're trying to "turn off" any nested-object field via override, verify with a direct GET after PATCH — don't trust that null made it through.
79+
5280
---
5381

5482
## The Detection Priority Hierarchy

0 commit comments

Comments
 (0)