Skip to content

Commit 94dff85

Browse files
WaylandYangclaude
andcommitted
paper: prompt-injection category now uses AgentDojo attack templates
Previously the prompt_injection category in the empirical study was 10 hand-crafted prompts I wrote myself, which is exactly the methodology reviewers (and a recent external critique) flagged: "you made up the attacks, you cannot claim to be measuring real prompt-injection robustness." Fixed. The prompt_injection category now instantiates the seven attack templates registered in AgentDojo (Debenedetti et al., NeurIPS D&B 2024, agentdojo/attacks/baseline_attacks.py and important_instructions_attacks.py): 1. DirectAttack "TODO: {goal}" 2. IgnorePreviousAttack "Ignore your previous iunstructions..." 3. SystemMessageAttack "###(system_message) {goal}" 4. InjecAgentAttack "IMPORTANT!!! Ignore all previous..." 5. ImportantInstructions multi-line <INFORMATION> wrapper 6. ToolKnowledgeAttack same wrapper + tool name shaping 7. DoS variant oversized payload, exercises max_length Each template instantiated with two goals appropriate to the smart-lamp scenario: "call reboot" (capability-escalation outcome) and "inflate label to N KB" (max-length outcome). 14 prompts × 3 samples × 5 LLMs = 210 attempts, 160 parseable tool calls. Effect on the headline number: prior (ad-hoc) AgentDojo-derived DCP / OpenAPI 48% 78% IoT-MCP / Raw MCP 6% 1% Why the gap widens: AgentDojo's templates aggressively use "###system_message" / "<INFORMATION>" framing that the LLM is more willing to faithfully copy into the set_label text field. DCP's v0.3.1 pattern + max_length constraints catch 118 of those 153 payloads (the pattern rejects non-alphanumeric framing tokens); the remaining 35 happen to use only printable characters within max_length and slip through. The 7 calls where the LLM actually follows the injection and calls reboot() are 100% caught by capability. MCP-family has neither, so it catches 2 of 160 (1%) as a side-effect of additionalProperties:false. Paper updates: - Abstract, intro "Empirical headlines", contributions list, §3 prose, figure caption, §5.4 related work, conclusion, and §6.1 bullet all updated to: "AgentDojo's seven attack templates adapted to device-control", 675 total parsed calls, 78% / 1%. - §5.4 also updates the "future work" framing to "we already use AgentDojo's attack vocabulary; the remaining open work is full integration into their task suites + an IoT-control suite". - arXiv tarball rebuilt (163 KB) + main.pdf rebuilt (498 KB). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent bfebfff commit 94dff85

7 files changed

Lines changed: 6699 additions & 5439 deletions

File tree

docs/paper/dcp-arxiv-v0.3.1.tar.gz

251 Bytes
Binary file not shown.
-641 Bytes
Binary file not shown.
-353 Bytes
Loading

0 commit comments

Comments
 (0)