Tracking Shuffled Objects

BIG-bench task for tracking object positions through a sequence of swaps.

unassessed

This page is a discovery lead. Nobody has yet assessed it against the catalogue contract, so it carries no disposition. Absence of evidence here is not evidence of staleness.

Category	reasoning
Subcategory	multi-step object tracking
Page status	active
Metric	accuracy
Direction	higher_is_better
Unit	percent
Dataset licence	Apache-2.0
Publisher	Google BIG-bench

What it measures

The task gives initial object positions and textual swaps, then asks for the final positions. It exercises symbolic tracking and multi-step reasoning.

Task format

Multiple-choice answer to a swap sequence.

Models reporting this benchmark

No model card in ModelSpec reports this benchmark yet.

Data

This page as JSON · Edit on GitHub