Portuguese multiple-choice questions from Unicamp (Convest) and USP (Fuvest) entrance exams, including image-linked items that most text-only harnesses drop.
unassessed
| Category | knowledge |
|---|---|
| Subcategory | Brazilian university-entrance multiple-choice exams (Portuguese) |
| Page status | active |
| Metric | exact_match (HELM); accuracy in the original paper |
| Direction | higher_is_better |
| Unit | % |
| Dataset size | 1422 |
| Publisher | University of Campinas (UNICAMP) / Portuguese-Benchmark-Datasets |
BLUEX tests whether a model can answer official Brazilian vestibular questions written in Portuguese. Items come from Unicamp’s Convest and USP’s Fuvest papers. Subjects span high-school fields such as mathematics, history, and languages. Many questions need an accompanying figure. Metadata flags image use, mathematical reasoning, and Brazil-specific knowledge. HELM’s scenario is text-only: it skips any item marked has_associated_images.
Multiple-choice. HELM uses joint letter answering with a Portuguese instruction and one worked example, scored by exact match. The 2023 paper also used few-shot prompts drawn from a different year of the same university.
No model card in ModelSpec reports this benchmark yet.