IMDb PT-BR (HELM)

HELM scenario that classifies Maritaca's Portuguese IMDb reviews as positivo or negativo; the scored test file has 5,000 balanced rows.

Also known as: imdb_pt, maritaca-ai/imdb_pt, IMDB PT-BR

unassessed

This page is a discovery lead. Nobody has yet assessed it against the catalogue contract, so it carries no disposition. Absence of evidence here is not evidence of staleness.

Category	reasoning
Subcategory	Brazilian Portuguese binary movie-review sentiment (Maritaca translation; HELM scenario)
Page status	unknown
Metric	exact_match
Direction	higher_is_better
Unit	%
Dataset size	5000
Publisher	Maritaca AI (Portuguese dump); Stanford CRFM HELM (scenario); Stanford NLP (English source dataset)

What it measures

imdb_ptbr asks a model to read a Portuguese movie review and label it positivo or negativo. HELM wraps Maritaca AI's Hub dump maritaca-ai/imdb_pt, a Portuguese translation of the Stanford Large Movie Review Dataset. The English source used only strongly polar reviews. The probe is Brazilian Portuguese sentiment classification, not English [imdb](imdb.md) and not HELM's contrast-set English IMDb scenario.

Task format

Binary sentiment generation in Portuguese. HELM's run spec prompts with two fixed in-context reviews, then "Resenha:" and "Classe:", and expects positivo or negativo. The scenario maps Hub labels 0/1 to those strings.

Models reporting this benchmark

No model card in ModelSpec reports this benchmark yet.

Data

This page as JSON · Edit on GitHub