Type: url
Original: https://arxiv.org/pdf/2605.25536
Status: generated
Summary
This tertiary review consolidates 30 secondary studies published between 2017 and 2025 on large language model-based code-generating tasks, synthesising their evidence against SWEBOK knowledge areas and an adaptation of the HELM evaluation framework. It reports that accuracy appears strong on standard benchmarks but is weakly supported for real-world generalisation, that robustness is fragile across tasks and configurations, that efficiency constraints are pervasive, and that toxicity and bias are under-reported. The dominant challenges identified concern economic feasibility, evaluation validity and socio-technical integration.