MultiPL-E: Perl

The Perl subset of MultiPL-E: HumanEval and MBPP function-completion problems translated into Perl and scored with pass@1.

unassessed

This page is a discovery lead. Nobody has yet assessed it against the catalogue contract, so it carries no disposition. Absence of evidence here is not evidence of staleness.
Categorycoding
Subcategorymultilingual code generation
Page statusactive
Metricpass@1
Directionhigher_is_better
Unit%
Dataset licenceMIT
PublisherNortheastern University Programming Research Lab (nuprl)

What it measures

This subset translates the HumanEval and MBPP prompts into Perl by rewriting each problem's function signature, docstring and tests with Perl syntax and typing, then asks the model to complete the function body in Perl. The underlying algorithmic problem is unchanged from the Python original; only the surface language differs.

Task format

Function completion in Perl: given a translated signature, docstring and (for HumanEval-derived items) doctests, the model generates a function body, which is compiled or interpreted with a real Perl toolchain inside a container and checked against translated unit tests.

Models reporting this benchmark

These figures come from the model cards, which carry one collection date per card and no per-score attribution. They are shown as reported, not as verified evidence.
ModelProviderScoreCard as of
GPT-5.1OpenAI63.52026-04
GPT-5.1 ChatOpenAI63.52026-04
GPT-5.1 CodexOpenAI63.52026-04
GPT-5.1 Codex MaxOpenAI63.52026-04
GPT-5.1 Codex miniOpenAI63.52026-04
GPT-5OpenAI62.82026-04
GPT-5 Chat (latest)OpenAI62.82026-04
GPT-5 MiniOpenAI62.82026-04
GPT-5 NanoOpenAI62.82026-04
GPT-5 ProOpenAI62.82026-04
GPT-5-CodexOpenAI62.82026-04
GPT-5.2OpenAI62.82026-04
GPT-5.2 ChatOpenAI62.82026-04
GPT-5.2 CodexOpenAI62.82026-04
GPT-5.2 ProOpenAI62.82026-04
GPT-5.3 Chat (latest)OpenAI62.82026-04
GPT-5.3 CodexOpenAI62.82026-04
GPT-5.3 Codex SparkOpenAI62.82026-04
GPT-5.4OpenAI62.82026-04
GPT-5.4 miniOpenAI62.82026-04
GPT-5.4 nanoOpenAI62.82026-04
GPT-5.4 ProOpenAI62.82026-04
Claude Opus 4Anthropic62.52026-04
Claude Opus 4.6Anthropic62.52026-04
o3OpenAI62.12026-04
Claude Opus 4.5Anthropic61.22026-04
Claude Opus 4.5 (latest)Anthropic61.22026-04
Claude Sonnet 4.6Anthropic60.52026-04
Claude Opus 4.1Anthropic60.22026-04
Claude Opus 4.1 (latest)Anthropic60.22026-04
Claude Sonnet 4Anthropic58.82026-04
Claude Sonnet 4.5Anthropic58.82026-04
Claude Sonnet 4.5 (latest)Anthropic58.82026-04
Qwen3-Coder 480B-A35B InstructAlibaba / Qwen Team58.82026-04
Gemini 2.5 ProGoogle DeepMind58.52026-04
GPT-4.1OpenAI58.22026-04
Claude Opus 4 (latest)Anthropic57.82026-04
Grok 4xAI57.52026-04
Grok 4 FastxAI57.52026-04
Grok 4 Fast (Non-Reasoning)xAI57.52026-04
Grok 4.1 FastxAI57.52026-04
Grok 4.1 Fast (Non-Reasoning)xAI57.52026-04
Grok 4.20 (Non-Reasoning)xAI57.52026-04
Grok 4.20 (Reasoning)xAI57.52026-04
Grok 4.20 Multi-AgentxAI57.52026-04
o4-miniOpenAI57.22026-04
o4-mini-deep-researchOpenAI57.22026-04
Claude Sonnet 4 (latest)Anthropic55.52026-04
GPT-4oOpenAI55.22026-04
GPT-4o (2024-05-13)OpenAI55.22026-04
GPT-4o (2024-08-06)OpenAI55.22026-04
GPT-4o (2024-11-20)OpenAI55.22026-04
Qwen3 235B-A22BAlibaba / Qwen Team55.22026-04
Claude Sonnet 3.7Anthropic54.52026-04
o3-miniOpenAI54.22026-04
DeepSeek V3.2 ExpDeepSeek53.82026-04
DeepSeek V3.2DeepSeek52.82026-04
DeepSeek R1 0528DeepSeek52.52026-04
DeepSeek R1 0528 NVFP4 v2NVIDIA52.52026-04
GPT-4 TurboOpenAI52.52026-04
Claude Sonnet 3.5Anthropic52.22026-04
Claude Sonnet 3.5 v2Anthropic52.22026-04
Claude Haiku 4.5Anthropic50.52026-04
Claude Haiku 4.5 (latest)Anthropic50.52026-04
DeepSeek R1DeepSeek50.22026-04
DeepSeek ReasonerDeepSeek50.22026-04
DeepSeek V3.1DeepSeek49.52026-04
Gemini 2.5 FlashGoogle DeepMind49.52026-04
Gemini 2.5 Flash ImageGoogle DeepMind49.52026-04
Gemini 2.5 Flash Image (Preview)Google DeepMind49.52026-04
Gemini 2.5 Flash LiteGoogle DeepMind49.52026-04
Gemini 2.5 Flash Lite Preview 06-17Google DeepMind49.52026-04
Gemini 2.5 Flash Lite Preview 09-25Google DeepMind49.52026-04
Gemini 2.5 Flash Preview 04-17Google DeepMind49.52026-04
Gemini 2.5 Flash Preview 05-20Google DeepMind49.52026-04
Gemini 2.5 Flash Preview 09-25Google DeepMind49.52026-04
Gemini 2.5 Flash Preview TTSGoogle DeepMind49.52026-04
Grok 3xAI49.22026-04
Grok 3 FastxAI49.22026-04
Grok 3 Fast LatestxAI49.22026-04
Grok 3 LatestxAI49.22026-04
Grok 3 MinixAI49.22026-04
Grok 3 Mini FastxAI49.22026-04
Grok 3 Mini Fast LatestxAI49.22026-04
Grok 3 Mini LatestxAI49.22026-04
Mistral Large (latest)Mistral AI48.52026-04
Mistral Large 2.1Mistral AI48.52026-04
Mistral Large 3Mistral AI48.52026-04
Qwen3 32BAlibaba / Qwen Team48.52026-04
Qwen3 32B AWQAlibaba / Qwen Team48.52026-04
Qwen3 32B NVFP4NVIDIA48.52026-04
DeepSeek V3 0324DeepSeek47.52026-04
GPT-4OpenAI46.52026-04
GPT-4.1 miniOpenAI46.52026-04
GPT-4.1 nanoOpenAI46.52026-04
Claude Opus 3Anthropic46.22026-04
DeepSeek ChatDeepSeek45.22026-04
DeepSeek V3DeepSeek45.22026-04
Llama 4 Maverick 17B 128E InstructMeta44.52026-04
Llama-4-Maverick-17B-128E-Instruct-FP8Meta44.52026-04
GPT-4o miniOpenAI44.22026-04
Qwen2.5 72B InstructAlibaba / Qwen Team44.22026-04
Claude Haiku 3.5Anthropic42.52026-04
Claude Haiku 3.5 (latest)Anthropic42.52026-04
Gemma 4 31BGoogle DeepMind42.52026-04
gemma 4 31B itGoogle DeepMind42.52026-04
gemma 4 31B it GGUFUnsloth42.52026-04
Gemma 4 31B IT NVFP4NVIDIA42.52026-04
Mistral Medium (latest)Mistral AI42.52026-04
Mistral Medium 3Mistral AI42.52026-04
Mistral Medium 3.1Mistral AI42.52026-04
Llama 4 Scout 17B 16EMeta41.22026-04
Llama 4 Scout 17B 16E InstructMeta41.22026-04
Llama-4-Scout-17B-16E-Instruct-FP8Meta41.22026-04
phi 4Microsoft40.82026-04
Gemma 4 26BGoogle DeepMind40.52026-04
Llama 3.3 70B Instruct NVFP4NVIDIA40.52026-04
Llama-3.3-70B-InstructMeta40.52026-04
Llama 3.1 70BMeta38.52026-04
Llama 3.1 70B InstructMeta38.52026-04
Mistral Small (latest)Mistral AI35.22026-04
Mistral Small 24B Instruct 2501Mistral AI35.22026-04
Mistral Small 3.1 24B Instruct 2503Mistral AI35.22026-04
Mistral Small 3.2Mistral AI35.22026-04
Mistral Small 3.2 24B Instruct 2506Mistral AI35.22026-04
Mistral Small 4Mistral AI35.22026-04
Mistral Small 4 119B 2603Mistral AI35.22026-04
Phi 4 mini instructMicrosoft30.52026-04
CodeLlama 34B Instruct hfMeta25.22026-04

Data

This page as JSON · Edit on GitHub