MultiPL-E: Rust

The Rust subset of MultiPL-E: HumanEval and MBPP function-completion problems translated into Rust and scored with pass@1.

unassessed

What it measures

This subset translates the HumanEval and MBPP prompts into Rust by rewriting each problem's function signature, docstring and tests with Rust syntax and typing, then asks the model to complete the function body in Rust. The underlying algorithmic problem is unchanged from the Python original; only the surface language differs.

Task format

Function completion in Rust: given a translated signature, docstring and (for HumanEval-derived items) doctests, the model generates a function body, which is compiled or interpreted with a real Rust toolchain inside a container and checked against translated unit tests.

Models reporting this benchmark

Data

This page as JSON · Edit on GitHub