Type: url
Original: https://arxiv.org/pdf/2403.08299
Status: generated
Summary
The paper introducing AutoDev, a Microsoft framework in which autonomous AI agents carry out user-defined software engineering objectives by performing file editing, retrieval, build, execution, testing and git operations inside a Docker-confined evaluation environment, under user-configured permissions and guardrails. On HumanEval it reports 91.5% Pass@1 for code generation and 87.8% Pass@1 for test generation with a single GPT-4 agent, with the passing generated tests reaching 99.3% coverage.