Type: url
Original: https://aclanthology.org/2025.acl-long.1266.pdf
Status: generated
Summary
This ACL 2025 long paper proposes ToolMaker, an agentic framework that autonomously turns a scientific paper’s public GitHub repository into an LLM-compatible tool: given a short task description, the repository URL and an example invocation, it installs dependencies into a reproducible Docker environment and writes a Python function, debugging it through a closed-loop self-correction cycle. It also introduces TM-Bench, a benchmark of 15 computational tasks, mostly from medicine and life sciences, with over 100 held-out unit tests. On it, ToolMaker correctly implements 12 of 15 tasks (80%), against 3 of 15 for the OpenHands software-engineering agent.