Skip to content

Commit bce4403

Browse files
dhruva-vapiclaude
andcommitted
docs(learnings): add 1000-char limit on function.description across tool types
Vapi enforces a hard 1000-character maximum on function.description for all tool types (function, apiRequest, transferCall, endCall, dtmf, voicemail, handoff, code). Over-limit tools don't fail at push time — they ship to the dashboard but the LLM degrades silently: tools stop being invoked at the right moment, the LLM picks shorter-description alternatives whose envelope fits the output budget, and platform-fired tool types (voicemail, dtmf) lose trigger-detection signal. Documents the diagnostic signal ("if a tool with a long, detailed description is being mis-fired, measure description length first"), an audit one-liner, and content guidance: drop strategy sections that duplicate the assistant prompt, drop long phrase lists for platform-fired tools (Vapi runs detection independently), drop anti-TTS-leakage META preambles (tool descriptions never reach TTS). Bonus: documents the function.name regex constraint (^[A-Za-z0-9_-]+$). Surfaced from a real Mudflap incident: end-on-voicemail at 1919 chars, dtmf at 1747 chars, and end-call-vapi-testing at 1100 chars were all silently over the limit; classifier symptoms (DTMF firing on humans, handoff not firing) compounded the over-limit signal degradation. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent 9cf6f64 commit bce4403

1 file changed

Lines changed: 37 additions & 0 deletions

File tree

docs/learnings/tools.md

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,43 @@ Non-obvious behaviors and silent defaults for Vapi tool types.
44

55
---
66

7+
## Common to All Tool Types
8+
9+
### `function.description` must be under 1000 characters
10+
11+
Vapi enforces a hard **1000-character maximum** on `function.description` across every tool type (`function`, `apiRequest`, `transferCall`, `endCall`, `dtmf`, `voicemail`, `handoff`, `code`). Tools with descriptions ≥ 1000 chars don't fail loudly at push time — they ship to the dashboard, but the LLM behavior degrades in ways that look like prompt or model bugs:
12+
13+
- The tool may stop being invoked at the right moment (or at all).
14+
- The LLM may invoke a *different* (cheaper-to-emit / shorter-description) tool whose envelope fits its remaining context budget.
15+
- For platform-fired tool types like `type: voicemail` and `type: dtmf`, an over-limit description may degrade the trigger-detection signal that the platform pipeline reads from the description metadata.
16+
17+
**Diagnostic signal:** if a tool with a long, detailed description (verbose WHEN-TO-CALL / STRATEGY / numbered phrase lists) is being mis-fired or ignored — measure the description length first, before changing the prompt or the model.
18+
19+
**Recommendation:**
20+
21+
- Validate description length at authoring time. A one-liner: `python3 -c 'import yaml,sys; print(len((yaml.safe_load(open(sys.argv[1])).get("function") or {}).get("description","")))' path/to/tool.yml`.
22+
- Keep descriptions **focused on the LLM-visible decision**: WHEN to call, WHEN NOT to call, the parameter shape. Drop:
23+
- **Strategy sections** that duplicate logic already in the assistant's system prompt (e.g., "STRATEGY FOR REACHING A HUMAN" duplicates IVR-handling rules from the prompt).
24+
- **Long phrase lists** for platform-fired tools like `type: voicemail`. Vapi's voicemail-detection pipeline runs independently of the LLM-visible description; a 38-numbered-phrase list adds noise without changing detection.
25+
- **Anti-TTS-leakage META preambles**. Tool descriptions are never voiced by the TTS pipeline — they reach the LLM as function-spec metadata, not assistant content. The "do not verbalize" guard is unnecessary and costs ~150–250 chars.
26+
- Audit existing tools when symptoms appear:
27+
```bash
28+
python3 -c '
29+
import yaml, glob
30+
for p in glob.glob("resources/*/tools/*.yml"):
31+
d = (yaml.safe_load(open(p)).get("function") or {}).get("description","") or ""
32+
if len(d) >= 1000: print(f"{p}: {len(d)} chars")
33+
'
34+
```
35+
36+
**Sweet spot:** 200–800 chars for a well-scoped tool. Above 800, audit for content that belongs in the assistant prompt instead.
37+
38+
### `function.name` matches `^[A-Za-z0-9_-]+$`
39+
40+
Tool names are validated against this regex by Vapi. Spaces, dots, slashes, parentheses, or unicode characters cause a 400 at push time. Use snake_case or camelCase (e.g. `end_call_vapi_testing`, `handoffToiFormSales`). The name is what the LLM emits in its function call, so keep it stable across config changes — renaming a tool invalidates any prompt rule that mentions the old name.
41+
42+
---
43+
744
## apiRequest Tools
845

946
### `body` is the single source of truth for the LLM schema

0 commit comments

Comments
 (0)