Type: url
Original: https://arxiv.org/pdf/2306.09896
Status: generated
Summary
The full text of an ICLR 2024 paper that studies self-repair for code generation, in which a model debugs and repairs its own code, with CodeLlama-13b-instruct, GPT-3.5 and GPT-4 on HumanEval and APPS. It finds that once the cost of repair is counted, gains are often modest and inconsistent, that spending budget on diverse initial samples matters more than extensive repair, and that the bottleneck is the quality of the model’s feedback on its own code: stronger-model or human feedback yields substantially larger gains.