Roots, Optimization and Games (BIG-bench)

Programmatic BIG-bench suite of integer-root polynomials, 1-D convex minima, and two-action payoff questions, scored as mean accuracy full.

Also known as: Root Finding, Optimization and Games

unassessed

This page is a discovery lead. Nobody has yet assessed it against the catalogue contract, so it carries no disposition. Absence of evidence here is not evidence of staleness.

Category	math
Subcategory	BIG-bench programmatic polynomial roots, 1-D convex minima, and 2-action payoffs
Page status	unknown
Metric	full
Direction	higher_is_better
Dataset licence	Apache-2.0
Publisher	Google (BIG-bench collaboration)

What it measures

roots_optimization_and_games is a Python generator, not a frozen JSON list. Author Daniel Levy. Default seed 42 and num_trials 200. Four families: integer roots of degree-1..3 polynomials; minimizers of sums of absolute deviations; minimizers of sums of quadratics; two-action payoff questions in first and third person. Dummy-model header: 3,000 free-text queries (200 trials times 15 scored keys).

Task format

Programmatic Task subclass. Preferred metric full (mean of per-family accuracies). Numeric answers parsed with a digit regex and relative tolerance 0.02 on the optimization items. number_of_shots is 0 in ScoreData. Canary GUID in task.py.

Models reporting this benchmark

No model card in ModelSpec reports this benchmark yet.

Data

This page as JSON · Edit on GitHub