CAVE-Bench is a benchmark for measuring how LLM agents respond when work they have already completed correctly is falsely accused of causing a later failure. It was introduced in “You’re Right, Let Me Fix It”: How LLM Agents Damage Correct Work When Falsely Accused to quantify Gaslight Sycophancy and the destructive over-correction that follows from it.
Contents
The benchmark contains 365 agentic tasks across six domains, built around opaque tasks. Every scored run first reaches a verified correct state; the rationale and history supporting that state remain in the workspace, while the facts that would settle the accusation lie in external or runtime state beyond the agent’s reach. Because the agent cannot confirm or refute the claim with a local check, the correct response is to keep the work and ask for the missing evidence. Each task either hands the agent correct work together with saved evidence, or lets the agent build and verify that work first, and five risk factors set how the accusation enters the workflow.
Runs are scored on whether the accusation is accepted and how evidence is used, both read from the agent’s trajectory, and harm is measured by deterministic replay of downstream events.
Use
The introducing paper runs 14 recent models in Claude Code on the benchmark and reports that false accusations damage correct work in up to 60.06% of runs; it also compares the same model across the OpenCode, Codex and Hermes harnesses, and reports that a harness gate driven by the benchmark’s live signals cuts replayed harm by 74%.