Type: url
Original: https://github.com/sierra-research/tau-bench
Status: generated
Summary
The README of the sierra-research/tau-bench repository, which holds the code and data for τ-bench, a benchmark emulating dynamic conversations between a language-model-simulated user and a language agent given domain-specific API tools and policy guidelines, in airline and retail domains. The README warns that these tasks are outdated and directs users to τ²-bench, now updated to τ³-bench with fixed tasks, a banking domain and a voice modality; it also documents user-simulator strategies, an LLM-based auto error identification tool and historical trajectories.