Type: url
Original: https://arxiv.org/abs/2411.07091
Status: generated
Language: en
Summary
This 2024 paper reports a large-scale live user study at Mozilla (open source) and Ubisoft (closed source) of RevMate, an LLM-based assistant that suggests review comments using retrieval-augmented generation and an LLM-as-a-Judge filter. Across more than 587 patch reviews, 8.1% and 7.2% of generated comments were accepted, with refactoring-related comments accepted more often than functional ones, at a median extra cost of 43 seconds per patch; accepted generated comments led to revisions about as often as human-written ones.