Type: url
Original: https://arxiv.org/pdf/2402.13521
Status: generated
Summary
The full text of an empirical study of whether test-driven development can be incorporated into LLM code generation, using a simple framework (TGen) that supplies human-written tests alongside the problem statement and runs a remediation loop on failing tests. With GPT-4 Turbo and Llama 3 on MBPP, HumanEval and a curated set of 1,100 CodeChef problems, it reports that supplying tests improves correctness by 9.15 to 29.57% and remediation adds 5.26 to 9.02% on the function-level benchmarks after validation with private tests.