Skip to content

Commit 57e66c6

Browse files
committed
docs: squad membersOverrides for multi-member post-call extraction
Document structuredDataPlan + fullMessageHistoryEnabled on squad membersOverrides, and when to prefer that over per-assistant plans.
1 parent cf07f81 commit 57e66c6

3 files changed

Lines changed: 75 additions & 2 deletions

File tree

docs/learnings/README.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -40,8 +40,8 @@ Gotchas and silent defaults for each resource type:
4040
|------|----------------|
4141
| [tools.md](tools.md) | apiRequest, function, transferCall, endCall, handoff, voicemail, dtmf, code tools; tool messages; strict mode |
4242
| [assistants.md](assistants.md) | Model defaults, voice, transcriber, firstMessage, outbound modes, voicemailMessage, hooks, idle messages, endpointing, interruption, analysis, artifacts, background sound, server messages, HIPAA, tool resolution |
43-
| [squads.md](squads.md) | Name uniqueness, tools:append, assistantDestinations, handoff context, contextEngineeringPlan, VM detection relay pattern, override merge order |
44-
| [structured-outputs.md](structured-outputs.md) | Schema type gotchas, assistant_ids, default models, target modes, KPI patterns |
43+
| [squads.md](squads.md) | Name uniqueness, tools:append, assistantDestinations, handoff context, contextEngineeringPlan, VM detection relay pattern, override merge order, `membersOverrides` structuredDataPlan + fullMessageHistory |
44+
| [structured-outputs.md](structured-outputs.md) | Schema type gotchas, assistant_ids, default models, target modes, KPI patterns, squad `membersOverrides` vs standalone SOs |
4545
| [simulations.md](simulations.md) | Personalities, evaluation comparators, chat-mode gotcha, missing references, full `/eval/simulation/*` API reference |
4646
| [webhooks.md](webhooks.md) | Default server messages, timeouts, unreachable servers, credential resolution, payload shape |
4747
| [voice-providers.md](voice-providers.md) | Per-provider voice block layout (Cartesia vs 11labs vs OpenAI/Azure/Rime/LMNT/Minimax/Neuphonic/SmallestAI) — saves 400s at push time |

docs/learnings/squads.md

Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -169,6 +169,67 @@ Later layers win on conflicts. `variableValues` from all layers are merged separ
169169

170170
---
171171

172+
## Squad-level post-call extraction via `membersOverrides` (multi-member squads)
173+
174+
**What you might expect:** Every squad member that should contribute to post-call KPIs needs its own `analysisPlan.structuredDataPlan` on the assistant file, or you attach standalone structured-output resources via `artifactPlan.structuredOutputIds`.
175+
176+
**What actually happens:** Vapi has no separate “squad structured output” resource. For squads, set **`membersOverrides.analysisPlan.structuredDataPlan`** on the squad YAML so the same extraction plan runs once at end-of-call for **all members** — including members that never reached a live conversation (silent classifier, voicemail-only leaver, etc.). The gitops engine pushes this as squad `membersOverrides` on the platform squad object.
177+
178+
**Critical companion flag:** If extraction must reason over speech from **more than one member** (e.g. classifier → voicemail leaver → conversational agent), also set:
179+
180+
```yaml
181+
membersOverrides:
182+
artifactPlan:
183+
fullMessageHistoryEnabled: true
184+
analysisPlan:
185+
structuredDataPlan:
186+
enabled: true
187+
messages:
188+
- role: system
189+
content: |-
190+
You will be given the full transcript of a multi-assistant call, the ended reason,
191+
and context from all squad members.
192+
193+
Extract fields based on what was said and how the call ended.
194+
Include squad-specific rules for voicemail-only or classifier-only endings.
195+
196+
Json Schema:
197+
{{schema}}
198+
199+
Only respond with the JSON.
200+
- role: user
201+
content: |-
202+
Here is the transcript:
203+
204+
{{transcript}}
205+
206+
Here is the ended reason of the call:
207+
208+
{{endedReason}}
209+
schema:
210+
type: object
211+
properties:
212+
call_outcome:
213+
type: string
214+
enum: [goal_met, not_interested, transfer, no_answer, callback]
215+
summary:
216+
type: string
217+
# ... other KPI fields
218+
```
219+
220+
Without `fullMessageHistoryEnabled: true`, structured-data extraction often sees only the **last active member’s** transcript slice. Early voicemail or triage audio is missing from the prompt → empty or wrong KPI fields (`label_not_filled` in QA tooling).
221+
222+
**Recommendation:**
223+
224+
1. **Single source of truth** — Define the plan once under `membersOverrides`. **Remove** duplicate `analysisPlan.structuredDataPlan` from individual member assistants to avoid drift (two prompts, or assistant-only plan winning on SDR calls but not on VM-only calls).
225+
2. **Pass `{{endedReason}}`** in the user message when outcomes depend on how the call ended (`voicemail`, silence timeout, transfer, etc.), not only spoken content.
226+
3. **Squad-specific classification rules** in the system message — e.g. map voicemail-only / never-reached-live-agent paths to `no_answer` (or your enum), and only use `interested` / `not_interested` when a live human had a substantive exchange.
227+
4. **Standalone structured outputs** (`structuredOutputs/*.yml` + `artifactPlan.structuredOutputIds`) are a different pipeline. You can use both, but do not assume they replace `structuredDataPlan` on squad members unless you link them explicitly.
228+
229+
See also [structured-outputs.md → Squad `membersOverrides` vs standalone structured outputs](structured-outputs.md#squad-membersoverrides-vs-standalone-structured-outputs).
230+
231+
---
232+
172233
## toolIds in assistantOverrides require UUIDs
173234

174235
`assistantOverrides.model.toolIds` in squad members must use **Vapi platform UUIDs**, not local filenames. The gitops engine resolves filenames to UUIDs for base assistant `model.toolIds`, but it does **not** resolve them inside squad `assistantOverrides`. If you use a local filename, the push will fail with `each value in toolIds must be a UUID`.

docs/learnings/structured-outputs.md

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -85,3 +85,15 @@ Structured outputs are the primary way to measure voice agent performance. Commo
8585
| `request_success_rate` | Aggregated per-request | Percentage of individual requests completed |
8686

8787
**Tip:** Start with 2–3 boolean KPIs (`call_successful`, `appointment_booked`) before adding more complex extraction. Each additional field increases extraction cost and latency.
88+
89+
---
90+
91+
## Squad `membersOverrides` vs standalone structured outputs
92+
93+
**What you might expect:** Linking structured outputs in `artifactPlan.structuredOutputIds` is enough for every call path in a squad.
94+
95+
**What actually happens:** Standalone structured-output resources run against the call transcript when listed on the assistant/squad artifact plan. **`analysisPlan.structuredDataPlan`** (inline schema + messages on the assistant or squad) is a separate end-of-call extraction path. In multi-agent squads, calls that end on an early member (voicemail leaver, silent classifier) often **never had** `structuredDataPlan` on that member — only the final conversational member did — so KPI fields stay empty even when the call succeeded.
96+
97+
**Recommendation:** For squad-wide KPIs that must populate on **every** ending (VM-only, classifier-only, and live-agent), use **`squad.membersOverrides.analysisPlan.structuredDataPlan`** plus **`membersOverrides.artifactPlan.fullMessageHistoryEnabled: true`**. Remove per-assistant duplicate plans. Keep standalone structured outputs for evals, dashboard analytics, or schemas you want versioned as separate resources.
98+
99+
Full YAML pattern and merge-order notes: [squads.md → Squad-level post-call extraction via `membersOverrides`](squads.md#squad-level-post-call-extraction-via-membersoverrides-multi-member-squads).

0 commit comments

Comments
 (0)