MultiPL-E: JavaScript

The JavaScript subset of MultiPL-E: HumanEval and MBPP function-completion problems translated into JavaScript and scored with pass@1.

unassessed

What it measures

This subset translates the HumanEval and MBPP prompts into JavaScript by rewriting each problem's function signature, docstring and tests with JavaScript syntax and typing, then asks the model to complete the function body in JavaScript. The underlying algorithmic problem is unchanged from the Python original; only the surface language differs.

Task format

Function completion in JavaScript: given a translated signature, docstring and (for HumanEval-derived items) doctests, the model generates a function body, which is compiled or interpreted with a real JavaScript toolchain inside a container and checked against translated unit tests.

Models reporting this benchmark

Data

This page as JSON · Edit on GitHub