MultiPL-E: Kotlin (unconfirmed)

A Kotlin subset is not present in the MultiPL-E paper, repository or dataset card checked for this page; what this key measures is not established.

unassessed

What it measures

This id is catalogued in this repository as a MultiPL-E language subset, but Kotlin does not appear among MultiPL-E's supported languages in any primary source checked: it is absent from the original paper's language list, from the per-language translator scripts in the nuprl/MultiPL-E repository's dataset_builder directory (which covers Ada, C++, C#, Clojure, D, Dart, Elixir, F#, Go, Haskell, Java, Julia, JavaScript, Lua, Luau, OCaml, PHP, Perl, Python, R, Ruby, Racket, Rust, Scala, Bash, Swift and TypeScript, but no Kotlin translator), and from the humaneval-*/mbpp-* configs published on the nuprl/MultiPL-E Hugging Face dataset card. What a `multipl_e_kotlin` score in this repository's model cards actually measures could not be established from these sources.

Task format

Not established. If a Kotlin port exists in some other fork, extension, or vendor-internal harness, it was not located among the primary MultiPL-E sources checked for this page.

Models reporting this benchmark

Data

This page as JSON · Edit on GitHub