GraphWalks BFS (256K-1M context)

GraphWalks' breadth-first-search task, scored only on prompts from the dataset's longest file, spanning roughly 256K to 1M tokens of context.

Also known as: GraphWalks BFS, 256K subset of 1M

unassessed

What it measures

This id covers GraphWalks' breadth-first-search (BFS) operation, scored specifically on prompts drawn from the openai/graphwalks dataset's longest file, `graphwalks_256k_to_1mil.parquet`, whose prompts range from about 256,000 up to roughly 1 million characters of context. Given a directed graph written as an edge list of hex-hash node names, a starting node and a target depth, the model must return exactly the set of nodes reachable at that depth -- not nodes at intermediate depths, and not the starting node itself. Because the graph is scattered across a prompt near the top of what even long-context models support, this variant specifically tests whether multi-hop traversal still works correctly at the harder end of a model's supported context length, not just at short range.

Task format

A prompt with three worked examples followed by a large directed graph and a "perform a BFS from node X with depth N" instruction, drawn only from the dataset's 256K-to-1M-token file. The model replies with "Final Answer: [node1, node2, ...]" on the prompt's last line.

Models reporting this benchmark

Data

This page as JSON · Edit on GitHub