Agentless is an approach to automatically solving software development problems that deliberately omits the autonomous agent. It is introduced in Agentless: Demystifying LLM-based Software Engineering Agents, whose authors describe it as an agentless approach and set it against the verbose and complex setup of agent-based approaches. In place of that setup, Agentless employs what the authors call a simplistic three-phase process — localization, repair and patch validation — and withholds two capabilities that define an agent in this context: the LLM does not decide future actions, and it does not operate with complex tools.
Usage
The term names both the specific system the paper builds and the contrast it is built to draw. The authors’ stated motivation is the question of whether complex autonomous software agents are really necessary, given that current LLMs have limited abilities while agent-based approaches are correspondingly complex — so “agentless” is defined by subtraction from the agent-based systems it is compared with, rather than by a mechanism of its own beyond the three phases.
The system itself is published as an open-source, MIT-licensed repository under the OpenAutoCoder organization on GitHub, whose README spells out what each phase does for a single issue:
- Localization is hierarchical — the fault is first narrowed to specific files, then to relevant classes or functions, and finally to fine-grained edit locations.
- Repair takes those edit locations and samples multiple candidate patches per bug, in a simple diff format.
- Patch validation selects regression tests to run and generates an additional reproduction test for the original error; the test results are used to re-rank the remaining patches and select one to submit.
The repository is set up to run on SWE-bench problems with an OpenAI API key. It announces a 1.0 release on 1 July 2024 and a 1.5 release on 28 October 2024, and reports that in December 2024 the project integrated Agentless with Claude 3.5 Sonnet for runs on SWE-bench Lite and SWE-bench Verified. Its v1.5.0 release carries the complete Agentless runs on both of those benchmarks together with preprocessed repository-structure information for each SWE-bench problem, and the repository also holds the authors’ manual classifications of SWE-bench Lite and the filtered SWE-bench Lite-S problems.
When It Applies
The paper presents Agentless as applicable to end-to-end software development problems of the kind agent-based systems target, and reports results on that basis: on SWE-bench Lite the authors state that Agentless achieves both the highest performance — 32.00%, 96 correct fixes — and low cost, $0.70, compared with all existing open-source software agents. Those are measured figures from one evaluation rather than standing properties of the approach.
What Agentless assumes is visible in what it leaves out. Because the LLM does not decide future actions or use complex tools, the three phases must be sufficient for the problem: the relevant code must be findable by localization, the fix expressible as a repair, and its correctness decidable by patch validation. The authors do not claim the approach supersedes agents — they frame it as highlighting the currently overlooked potential of a simple, interpretable technique in autonomous software development, and state the hope that it will help reset the baseline, starting point and horizon for autonomous software agents and inspire future work in that direction.
The evidence behind it is one paper’s own evaluation on one benchmark, together with that paper’s finding that the benchmark itself contains problems with exact ground truth patches or insufficient and misleading issue descriptions — which is why the same authors construct SWE-bench Lite-S as a filtered basis for more rigorous comparison.
Related Terms
- SWE-bench Lite-S — the filtered benchmark the same paper constructs for more rigorous comparison
- SWE-Bench — the benchmark family the repository is set up to run on