Reordering

Three synthetic subtasks ask models to reconstruct ordered sentences from scrambled words, characters or swaps.

Also known as: BIG-bench Undo Permutation, Reordering

unassessed

This page is a discovery lead. Nobody has yet assessed it against the catalogue contract, so it carries no disposition. Absence of evidence here is not evidence of staleness.

Category	reasoning
Subcategory	word, character and swap-order reconstruction
Page status	active
Metric	multiple_choice_grade accuracy
Direction	higher_is_better
Unit	%
Dataset size	300
Publisher	BIG-bench collaboration

What it measures

Reordering tests sequential structure through scrambled words, scrambled characters including spaces, and sequences of word swaps.

Task format

Multiple choice across reorder_words, reorder_chars and reorder_byswap.

Models reporting this benchmark

No model card in ModelSpec reports this benchmark yet.

Data

This page as JSON · Edit on GitHub