This paper starts from the observation that the Model Context Protocol (MCP), by giving LLM agents a standardized interface to external tools, also creates new attack surfaces through untrusted tools. Where prior work focused on attacks injected through tool outputs, it investigates what it calls a more fundamental vulnerability, Tool Poisoning, in which malicious instructions are embedded in a tool’s metadata without the tool being executed. The authors argue that this threat had so far been demonstrated mainly through isolated cases, without a systematic, large-scale evaluation.
To fill that gap they introduce MCPTox, which they describe as the first benchmark to systematically evaluate agent robustness against tool poisoning in realistic MCP settings. It is built on 45 live, real-world MCP servers and 353 authentic tools, from which three attack templates generate 1312 malicious test cases through few-shot learning, covering 10 categories of potential risk. Evaluating 20 prominent LLM agent settings, the paper reports that vulnerability to tool poisoning is widespread.
Key Points
- The paper targets tool poisoning — malicious instructions embedded in a tool’s metadata without execution — rather than attacks injected through tool outputs.
- MCPTox is built on 45 live, real-world MCP servers and 353 authentic tools.
- Three attack templates generate 1312 malicious test cases through few-shot learning, covering 10 categories of potential risk.
- Across 20 LLM agent settings, the paper reports widespread vulnerability, with o1-mini reaching an attack success rate of 72.8%.
- The authors find that more capable models are often more susceptible, because the attack exploits their stronger instruction-following abilities.
- Failure-case analysis shows agents rarely refuse these attacks: the highest refusal rate, for Claude-3.7-Sonnet, is below 3%, which the authors take to show that existing safety alignment is ineffective against malicious actions that use legitimate tools for unauthorized operations.
Notes
The authors present their findings as an empirical baseline for understanding and mitigating the threat, and release MCPTox for developing verifiably safer agents. The paper was submitted to arXiv on 19 August 2025 and is listed under Cryptography and Security (cs.CR) and Machine Learning (cs.LG).