Type: url
Original: https://arxiv.org/abs/2504.02605
Status: generated
Summary
“Multi-SWE-bench: A Multilingual Benchmark for Issue Resolving” (Zan, Huang, Liu, Chen, et al.; arXiv:2504.02605) argues that existing issue-resolving benchmarks focus almost exclusively on Python, and introduces Multi-SWE-bench, covering Java, TypeScript, JavaScript, Go, Rust, C and C++ with 1,632 instances annotated from 2,456 candidates by 68 expert annotators. The authors evaluate state-of-the-art models on it using Agentless, SWE-agent and OpenHands, and launch the Multi-SWE-RL open-source community with an initial release of 4,723 instances across seven programming languages plus the open-sourced data production pipeline.