The post starts from a design problem its author says dominated building Orange Codens: when the reviewer is AI and the fixer is AI, a loop forms easily. Orange reviews a pull request, hands a finding to another agent that opens a fix PR, that fix PR is reviewed in turn, produces new findings, and so on. Where a human reviewer stops at “good enough”, the post argues, an AI reviewer does not, and every round costs LLM tokens — so termination and bounded cost have to be guaranteed by the architecture, not by how capable the model is.
The post then walks through the mechanisms built into the product, with excerpts of its code, and two incidents in which the pipeline, built to that design, still failed to converge — the first of them observed in an end-to-end run. It closes by listing the resulting cuts and noting that the product includes all of them; it is written by the product’s builder about their own product.
Key Points
- There is no single place to cut the loop: when to hand off, whose PRs are eligible, whether a finding can be dispatched twice, and when the review itself converges each need their own mechanism.
- Handoff of a fix task fires only when a PR is merged, not when a finding is created; before merge the reviewer only posts suggestions, and a PR closed without merging discards its queued handoffs, so an unmerged PR has no downstream cost.
- PRs authored by the fixing bots are still reviewed but never trigger an automatic handoff, which the author calls the direct break in the loop; findings from reviews of the fixing agent’s own verify runs are likewise excluded.
- A finding that has been handed off once is never dispatched again, preventing two fix tasks for the same problem.
- A finding a human explicitly queued is dispatched even on a bot PR and even with automatic handoff off: explicit human intent overrides the loop guard.
- Incident, in an end-to-end run: dispatching one fix task per finding let three findings in the same file deadlock, because each fix PR was blocked by the reviewer’s carry-over findings on the others. The fix was to coalesce findings for the same file and the same target agent into one task.
- Incident: reviews of bot fix PRs kept raising new high-severity findings on the newly written code, producing four consecutive request-changes rounds and escalation at the iteration cap. The fix was to block bot fix PRs only on unresolved high-severity carry-over findings or new blockers, recording other new findings but approving the PR; review of human PRs was left unchanged.
- The general lesson drawn is that “a perfectionist reviewer never converges”, so the review bar must be designed so the review can stop — presented as the author’s conclusion from operating this product, not as a measured result.
Context
The post is a first-person engineering account from a product’s builder, and ends by pointing to the product. Its argument that termination must come from the architecture is one answer to the problem described on Review Loop Non-Convergence; other writing on AI reviewing AI-authored changes is gathered under Closed-Loop AI Review and Agentic Code Review.