A bilingual tool-use benchmark that scores instruction, planning, reasoning, retrieval, understanding and review as separate steps rather than one success bit.
unassessed
| Category | agentic |
|---|---|
| Subcategory | step-wise LLM tool utilization |
| Page status | active |
| Metric | overall (mean of subset metrics) |
| Direction | higher_is_better |
| Dataset size | 23305 |
| Dataset licence | Apache-2.0 |
| Publisher | University of Science and Technology of China / Shanghai AI Laboratory / Tsinghua University / Jilin University / OpenCompass |
T-Eval tests whether a model can use tools by grading each sub-skill on its own. Given a query and a tool list, the model must plan, pick tools, fill parameters, emit a call, and review the tool response. The authors argue that scoring only the final answer hides which of those steps failed. English and Chinese prompts both exist. This is not a coding benchmark and not ToolBench's end-to-end win rate, though the paper plots the two against each other.
Multi-turn chat with system/user/assistant roles and a tool schema in the prompt. OpenCompass uses `ChatInferencer` and `TEvalEvaluator` on local JSON files named `instruct_v1`, `plan_json_v1`, `plan_str_v1`, and similar. Hugging Face `lovesnowbest/T-Eval` currently ships v2 files (`instruct_v2.json`, ...).
No model card in ModelSpec reports this benchmark yet.