Terminal-Bench 3.0

Terminal-Bench 3.0 is the v3.0.0 Harbor dataset release with task instructions, environments, tests and solutions.

unassessed

This page is a discovery lead. Nobody has yet assessed it against the catalogue contract, so it carries no disposition. Absence of evidence here is not evidence of staleness.

Category	agentic
Subcategory	Versioned agent execution on real terminal tasks
Page status	unknown
Metric	task success verified by tests
Direction	higher_is_better
Unit	%
Dataset size	74
Dataset licence	Apache-2.0
Publisher	Harbor Framework

What it measures

Terminal-Bench 3.0 is the v3.0.0 Harbor dataset release with task instructions, environments, tests and solutions. The benchmark gives an agent an English task, a terminal environment and a verification procedure; success depends on completing the task and passing its tests.

Task format

Agent interacts with a containerized terminal; task-specific tests determine success.

Models reporting this benchmark

No model card in ModelSpec reports this benchmark yet.

Data

This page as JSON · Edit on GitHub