A critic model is a model trained to judge a coding agent’s attempt rather than to produce one. In the form OpenHands describes in Learning to Verify AI-Generated Code, it is a small, fast model that scores an agent’s trajectory — the conversation, tool calls and actions of an attempt — and that score is then used to decide whether to continue, stop or refine, to pick among multiple attempts, and to collect lightweight feedback that improves the agent over time. OpenHands presents its critic as a trajectory-level verifier and the first layer of what it calls a verification stack.
Usage
OpenHands’ post notes that earlier academic work had already trained critic models on benchmark datasets such as SWE-bench and SWE-Gym, where verified rewards such as unit tests make it convenient to label each attempt as correct or not. Its own argument is that such rewards usually do not exist in real-world, human-in-the-loop development, where goals evolve and success signals are sparse or delayed, and it reports that critics trained only on benchmark traces translated poorly to production outcomes. OpenHands instead trains its critic on real-world user–agent interactions: each interaction is split into segments, every segment is annotated with rubric features observable in the trace, and a small subset is grounded in sparse outcome proxies such as whether a pull request was merged and whether the code survived. By OpenHands’ own evaluation, code survival made the stronger training signal of the two.
The post describes two ways it uses the score at inference time: best-of-N selection, reranking several attempts and keeping the best, and early stopping, accepting an attempt as soon as its score clears a threshold. Because the model is small, OpenHands says it is fast and cheap enough to run during an interactive session.