Pre-Flight

300 multiple-choice questions on ICAO, FAA, and airport ground-ops knowledge, scored by Inspect Evals accuracy.

Also known as: Pre-Flight, inspect_evals/pre_flight, pre-flight-06

unassessed

This page is a discovery lead. Nobody has yet assessed it against the catalogue contract, so it carries no disposition. Absence of evidence here is not evidence of staleness.

Category	knowledge
Subcategory	multiple-choice aviation operations, ICAO/FAA, and airport ground-ops knowledge
Page status	active
Metric	accuracy
Direction	higher_is_better
Unit	%
Dataset size	300
Dataset licence	MIT
Publisher	Airside Labs, with Mahino Research; distributed in Inspect Evals (UK AI Security Institute)

What it measures

Pre-Flight tests whether a language model knows aviation operational rules well enough to pick the right option on a written exam-style item. Questions cover airport ground operations, ICAO annexes and rules of the air, US FAA material, general aviation trivia, and multi-step stand-clearing scenarios. English only. Developed by Airside Labs. It is not a flight simulator, not a tool-using agent task, and not a vision exam: the scored set is text.

Task format

inspect_ai Task with multiple_choice solver and choice scorer. Hugging Face dataset AirsideLabs/pre-flight-06 split test, pinned to git revision 439d2d118fed7d9b009c1f87b9eb1205ab94766e. Each row has id, input, choices, and target letter. Many items include a fifth choice "no suitable option". eval.yaml version 2-A, group Knowledge, 300 samples. No task parameters.

Models reporting this benchmark

No model card in ModelSpec reports this benchmark yet.

Data

This page as JSON · Edit on GitHub