Type: url
Original: https://arxiv.org/abs/2512.12730
Status: generated
Summary
The arXiv abstract page for NL2Repo-Bench, a benchmark that evaluates whether coding agents can build a complete, installable Python library from a single natural-language requirements document and an empty workspace. The authors report that long-horizon repository generation remains largely unsolved, with even the strongest agents below 40% average test pass rates, and identify failure modes such as premature termination, loss of global coherence and inadequate planning.