Type: url
Original: https://arxiv.org/pdf/2406.13352
Status: generated
Summary
“AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents” (Debenedetti, Zhang, Balunovic, Beurer-Kellner, Fischer, Tramèr; NeurIPS 2024) introduces AgentDojo, an extensible benchmark of 4 tool-use environments, 97 user tasks, 27 injection tasks, and 629 security test cases, and uses it to evaluate tool-calling LLM agents’ vulnerability to prompt injection attacks and the effectiveness of several proposed defenses, finding that more capable models tend to be easier to attack and that a simple tool-filtering defense lowers the attack success rate the most among the defenses tested.