This paper argues that because large language models can be flexibly modulated via natural language prompts, LLM-integrated applications blur the line between data and instructions, opening a new attack surface beyond the assumption that a user is the one directly prompting the model. It introduces Indirect Prompt Injection: strategically planting adversarial prompts in data that an LLM-integrated application is likely to retrieve, letting an attacker remotely exploit the application without any direct interface to it.
The paper derives a taxonomy, from a computer security perspective, to systematically investigate the resulting impacts and vulnerabilities, and demonstrates the attacks’ practical viability against real-world systems.
Key Points
- The paper’s taxonomy of impacts and vulnerabilities from indirect prompt injection includes data theft, worming, and information ecosystem contamination, among other novel security risks.
- The attacks were demonstrated as practically viable against real-world systems, including Bing’s GPT-4-powered Chat and code-completion engines, as well as against synthetic applications built on GPT-4.
- The paper shows that processing a retrieved, injected prompt can act as arbitrary code execution from the application’s perspective, manipulating the application’s functionality and controlling whether and how other APIs are called.
- The paper states that, despite increasing integration and reliance on LLMs, effective mitigations for these threats were lacking at the time of writing.
Notes
This entry is based on the paper’s arXiv abstract page rather than its full text; the summary above reflects only what the abstract itself states, not the paper’s full methodology or detailed case-study results.