MultiPL-E: Go

The Go subset of MultiPL-E: HumanEval and MBPP function-completion problems translated into Go and scored with pass@1.

unassessed

What it measures

This subset translates the HumanEval and MBPP prompts into Go by rewriting each problem's function signature, docstring and tests with Go syntax and typing, then asks the model to complete the function body in Go. The underlying algorithmic problem is unchanged from the Python original; only the surface language differs.

Task format

Function completion in Go: given a translated signature, docstring and (for HumanEval-derived items) doctests, the model generates a function body, which is compiled or interpreted with a real Go toolchain inside a container and checked against translated unit tests.

Models reporting this benchmark

Data

This page as JSON · Edit on GitHub