You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
A read-through audit of the paper after the v0.3.1 changes surfaced
several stale or weak passages. Fixed in three priority tiers.
P0 (factual / self-contradictory):
- Introduction's "we make no claim of empirical superiority...will be
released upon paper acceptance" paragraph was directly contradicted
by the §3 empirical study and §4 IoT-MCP latency A/B that v0.3.1
added. Rewrote it as a plain pointer to the public repo + PyPI
package; the empirical content now lives in its own contribution
bullet (item 5) and in the new "Empirical headlines" paragraph.
- Removed a misplaced \cite{wot_td_2025} that had been copy-pasted
into the wrong sentence.
- Section 6 "deferred to v0.3 and beyond" -> "deferred to v0.4 and
beyond" (v0.3.1 is on PyPI).
P1 (impact / signal):
- Added "Empirical headlines" paragraph at the end of §1 that pulls
the two big numbers (100% capability rejection across 295 calls;
5 µs latency delta vs IoT-MCP wire) up front, instead of leaving
them for §3 to surface.
- Contributions list now has an explicit "empirical study" bullet
with the headline numbers and method (DeepSeek V3 + Qwen 2.5-72B
via SiliconFlow, six adversarial categories).
P2 (related work / conclusion / citations):
- §5.4 now cites AgentDojo, InjecAgent, ToolBench as the standard
LLM agent / tool-use safety benchmarks, with a sentence on why
this paper used its own corpus instead (device-control idioms
the general benchmarks don't cover) and a note that folding them
in is the obvious next step.
- Conclusion rewritten in three paragraphs: design contribution,
empirical contribution (the part we didn't expect to find), and
the call for collaborators. The empirical paragraph names the
100% / 5 µs numbers again so the reader leaves with them.
- refs.bib gains agentdojo, injecagent, toolbench entries.
PDF grows from 14 to 15 pages. arXiv tarball rebuilt against this
polish (same v0.3.1 release; tarball replaced in place).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Copy file name to clipboardExpand all lines: docs/paper/refs.bib
+24Lines changed: 24 additions & 0 deletions
Original file line number
Diff line number
Diff line change
@@ -180,3 +180,27 @@ @article{llm_iot_survey
180
180
year = {2025},
181
181
url = {https://arxiv.org/abs/2505.17586}
182
182
}
183
+
184
+
@inproceedings{agentdojo,
185
+
title = {{AgentDojo}: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for {LLM} Agents},
186
+
author = {Debenedetti, Edoardo and Zhang, Jie and Balunovi{\'c}, Mislav and Beurer-Kellner, Luca and Fischer, Marc and Tram{\`e}r, Florian},
187
+
booktitle = {NeurIPS Datasets and Benchmarks Track},
188
+
year = {2024},
189
+
url = {https://arxiv.org/abs/2406.13352}
190
+
}
191
+
192
+
@article{injecagent,
193
+
title = {{InjecAgent}: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents},
194
+
author = {Zhan, Qiusi and Liang, Zhixiang and Ying, Zifan and Kang, Daniel},
195
+
journal = {ACL Findings},
196
+
year = {2024},
197
+
url = {https://arxiv.org/abs/2403.02691}
198
+
}
199
+
200
+
@inproceedings{toolbench,
201
+
title = {{ToolLLM}: Facilitating Large Language Models to Master 16000+ Real-world {APIs}},
202
+
author = {Qin, Yujia and Liang, Shihao and Ye, Yining and Zhu, Kunlun and Yan, Lan and Lu, Yaxi and Lin, Yankai and Cong, Xin and Tang, Xiangru and Qian, Bill and Zhao, Sihan and Hong, Lauren and Tian, Runchu and Xie, Ruobing and Zhou, Jie and Gerstein, Mark and Li, Dahai and Liu, Zhiyuan and Sun, Maosong},
0 commit comments