ReAct is a prompting technique, proposed by Yao et al. in ReAct: Synergizing Reasoning and Acting in Language Models, that augments an agent’s action space with a “language space” of free-form thoughts alongside its ordinary task-specific actions. A thought does not itself act on the external environment and produces no observation feedback; instead, it composes useful information from the current context — such as decomposing a goal, tracking progress, or noting an exception — and updates the context to support later reasoning or acting. The technique is applied by prompting a frozen large language model with a handful of in-context examples, each a human-written trajectory of thoughts, actions, and environment observations, without any additional model training.
Usage
The technique is applied by writing the in-context examples by hand, one set per task, and the authors published theirs as an MIT-licensed repository of GPT-3 prompting code — a notebook per task, with a small Wikipedia environment supplying the search-and-lookup actions. That repository directs readers who want to apply ReAct to tasks beyond the four it covers to LangChain’s zero-shot ReAct agent rather than to its own prompts.
How far the technique carries across models is read differently by the two documents. The paper’s
appendix compares PaLM-540B with GPT-3 (text-davinci-002) on two tasks, finds GPT-3 ahead on
both, and reads that as evidence that ReAct prompting is effective across different large language
models. The repository’s table covers four tasks and has GPT-3 ahead on two and behind on the other
two, which it summarises as the two models being better at different tasks.
When It Applies
The introducing paper applies ReAct to tasks with distinct action spaces and reasoning needs: for knowledge-intensive tasks (multi-hop question answering, fact verification) it alternates thought and action steps densely, interacting with a simple Wikipedia search/lookup interface to retrieve supporting information; for interactive decision-making tasks (a text-based household simulation and an online-shopping environment) it lets thoughts appear more sparsely, at the points in a trajectory where they are most useful, with the language model itself deciding when to think versus act. The paper reports that ReAct consistently outperformed the acting-only baseline; against reasoning-only chain-of-thought it came out ahead on fact verification but slightly behind on multi-hop question answering, and the paper’s strongest results on those two tasks came from combining ReAct with self-consistency reasoning rather than from ReAct alone. It also reports that an ablation using dense, externally-triggered thoughts modeled on a prior “Inner Monologue” method underperformed ReAct’s sparser, more flexible thought pattern on the household-simulation task. The paper’s own stated limitation is that tasks with large action spaces need more demonstrations to learn well, which can exceed the input-length limits of in-context prompting.
Two further results in the paper bear on how the technique is used beyond the prompting setup, and the authors restate both in their own Google Research blog post. The first is the fine-tuning answer to the limitation just named: successful trajectories generated by the ReAct-prompted PaLM-540B model are used to fine-tune smaller PaLM-8B and PaLM-62B models, which avoids annotating reasoning traces by hand at scale, and ReAct is reported as the best-performing method under fine-tuning. The second is that a human can intervene in a trajectory — editing two of the agent’s thoughts, removing a hallucinating one and adding a hint, was enough to change its subsequent behaviour and let it complete a household-simulation task it had otherwise failed, which the authors present as a form of human-machine collaboration made practical by the fact that only a few thoughts need editing.