τ-bench

Scores whether a tool-using agent can hold a policy-following conversation with a simulated customer and leave the backend in the right state.

Also known as: tau-bench, tau bench

unassessed

What it measures

τ-bench asks a language agent to act as a customer-service representative in a retail or an airline domain: it chats with a user that is itself simulated by an LLM, calls domain-specific API tools to look up and change records, and must follow a written policy document. The user pursues its own goal and only reveals what it would naturally reveal, so the agent has to ask questions rather than execute a fixed call sequence.

Task format

multi-turn text dialogue interleaved with structured tool calls, against a simulated user model

Models reporting this benchmark

Data

This page as JSON · Edit on GitHub