Korean multiple-choice exam of cultural and linguistic knowledge: 1,995 questions in eleven categories, drawn from official exams and textbooks.
unassessed
| Category | knowledge |
|---|---|
| Subcategory | Korean cultural and linguistic multiple-choice QA from exams and textbooks |
| Page status | active |
| Metric | accuracy and length-normalized accuracy (acc, acc_norm) |
| Direction | higher_is_better |
| Unit | % |
| Dataset size | 1995 |
| Publisher | KAIST (School of Computing and GSAI) |
click is EleutherAI lm-evaluation-harness's group for CLIcK (Cultural and Linguistic Intelligence in Korean). The model reads a Korean question, often with a short context, and picks a lettered choice. Items test Korean language (textual, grammatical, functional knowledge) and Korean culture (society, tradition, politics, economy, law, history, geography, popular culture). Sources are official exams plus GPT-4 questions written from the KIIP textbook and then validated. This is a Korean-centric knowledge test, not a translated English quiz and not [korbench](korbench.md) (which is English knowledge-orthogonal reasoning despite the similar id).
Multiple choice in Korean. lm-eval output_type is multiple_choice. The Korean prompt in utils.get_context always lists A–D. Scoring helpers get_choices and get_target use A–E when the example id contains CSAT, else A–D. The Hugging Face split used as both test_split and fewshot_split is named train. Groups: click (all 11), click_lang (3), click_cul (8).
No model card in ModelSpec reports this benchmark yet.