{
 "body": "\n## What it measures\n\nTweetSentBR classifies Brazilian Portuguese tweets, drawn from television-show discussion on Twitter, as Positive, Negative, or Neutral. The original corpus paper reports tweets about talk shows, reality shows, and variety programs collected between January and July 2017.\n\n## How it is scored\n\nHELM constructs one correct reference label per tweet and evaluates single-label classification, mapping the English class names to Portuguese outputs (Positivo, Negativo, Neutro). The original 2018 paper reports 80.99% F-measure for a binary (positive/negative) classifier as its own headline result; that number is not directly comparable to the three-way accuracy HELM reports, since the label set and evaluation differ.\n\n## Dataset and licence\n\nThe original corpus (Brum and Volpe Nunes, 2018) contains 15,000 annotated tweets, split 12,999 train / 2,001 test, labelled by seven native-speaker annotators with label shares of roughly 44% positive, 26-29% neutral, and 29% negative in each split. Because Twitter's terms restrict redistribution of tweet text, the original release ships only tweet IDs, requiring re-download via the Twitter API. The Hugging Face datasets-server reports 75 train rows and 2,010 test rows (2,085 total) for `eduagarcia/tweetsentbr_fewshot`, the differently sized and pre-formatted mirror the HELM scenario actually loads; its own licence terms were not established.\n\n## Who publishes it\n\nTweetSentBR was introduced by Henrico Brum and Maria das Gra\u00e7as Volpe Nunes of NILC, University of S\u00e3o Paulo, published at LREC 2018. Stanford CRFM maintains the HELM few-shot scenario that packages a re-split, pre-formatted version of the corpus from `eduagarcia/tweetsentbr_fewshot`.\n\n## Lineage\n\nThis page documents the HELM few-shot scenario built on the original TweetSentBR corpus. The scenario's 75/2,010 split does not match the original paper's 12,999/2,001 split, so scores from the two should not be merged. No further predecessor or successor was established.\n\n## Saturation and contamination\n\nSaturation is unknown; no current leaderboard or ceiling analysis was established. Contamination risk is medium: the underlying tweets were collected in 2017 and published in 2018, and the HELM scenario's mirror redistributes full tweet text rather than IDs only, so the text is old and accessible enough to plausibly have entered training corpora, even though no specific leakage report was found.\n\n## How to run it\n\nRun HELM scenario `tweetsentbr`; it loads train and test splits and maps labels to Positivo, Negativo, and Neutro. Record dataset revision.\n\n## Reading the numbers\n\nA strong score indicates sentiment classification ability on Brazilian Portuguese tweets. It does not measure broader Portuguese language understanding or sentiment outside this corpus. Compare label mapping and few-shot examples.\n",
 "build": {
  "built_at": "2026-09-09T16:56:50+00:00",
  "commit": "0a599558854c0e238c03a0f0d725239cb28f9d11",
  "eligibility_as_of": "2026-09-09"
 },
 "disposition": {
  "canonical_id": "tweetsentbr",
  "reasons": [],
  "status": "unassessed",
  "verified_results": []
 },
 "models_covered": [],
 "page": {
  "aliases": [
   "tweetSentBR"
  ],
  "category": "domain",
  "contamination": {
   "note": "The corpus dates from 2017-2018 and full tweet text is redistributed via the eduagarcia/tweetsentbr_fewshot mirror on Hugging Face, so it is old enough and accessible enough to plausibly appear in training data, though no publisher or community report of leakage was found.",
   "risk": "medium"
  },
  "dataset": {
   "languages": [
    "Brazilian Portuguese"
   ],
   "license": "",
   "modalities": [
    "text"
   ],
   "public_test_set": true,
   "size": 2085,
   "size_note": "75 train rows and 2,010 test rows (2,085 total), per the Hugging Face datasets-server for eduagarcia/tweetsentbr_fewshot; the HELM scenario code itself only says 75 training samples and 'all' of the 2,000-plus test instances.",
   "splits": "train, test",
   "url": "https://huggingface.co/datasets/eduagarcia/tweetsentbr_fewshot"
  },
  "freshness": {
   "luna-new-002": null,
   "luna-new-002 (Codex coordinated)": null,
   "researched": "2026-09-08",
   "researched_by": "GPT-5.6 Luna",
   "reviewed": "2026-09-08",
   "reviewed_by": "Claude Sonnet 5 independent review"
  },
  "harness": {
   "bigbench": "",
   "helm": "tweetsentbr",
   "inspect_evals": "",
   "lm_eval": "",
   "opencompass": "",
   "other": ""
  },
  "id": "tweetsentbr",
  "last_updated": "",
  "leaderboard_url": "",
  "lineage": {
   "family": "",
   "predecessor": "",
   "successors": [],
   "variants": []
  },
  "measures": "Tweets are labeled Positive, Negative, or Neutral by multiple annotators.",
  "metric": {
   "baseline_note": "",
   "direction": "higher_is_better",
   "human_baseline": null,
   "max_score": 100,
   "name": "accuracy",
   "random_baseline": null,
   "unit": "percent"
  },
  "name": "TweetSentBR",
  "page_kind": "benchmark",
  "paper": {
   "arxiv": "",
   "title": "Building a Sentiment Corpus of Tweets in Brazilian Portuguese",
   "url": "https://aclanthology.org/L18-1658/",
   "year": 2018
  },
  "publisher": {
   "authors": [
    "Henrico Brum",
    "Maria das Gra\u00e7as Volpe Nunes"
   ],
   "org": "NILC, University of S\u00e3o Paulo (original corpus); Stanford CRFM HELM (few-shot scenario)",
   "url": "https://github.com/brasileiras-pln/tweetSentBR"
  },
  "released": "2018",
  "repo_url": "https://github.com/stanford-crfm/helm",
  "saturation": {
   "as_of": "",
   "note": "No current standalone leaderboard was established.",
   "status": "unknown",
   "top_score": null
  },
  "sources": [
   {
    "accessed": "2026-09-08",
    "title": "HELM TweetSentBR scenario",
    "url": "https://raw.githubusercontent.com/stanford-crfm/helm/main/src/helm/benchmark/scenarios/tweetsentbr_scenario.py"
   },
   {
    "accessed": "2026-09-08",
    "title": "Building a Sentiment Corpus of Tweets in Brazilian Portuguese (Brum and Volpe Nunes, LREC 2018)",
    "url": "https://aclanthology.org/L18-1658/"
   },
   {
    "accessed": "2026-09-08",
    "title": "Building a Sentiment Corpus of Tweets in Brazilian Portuguese, HTML version",
    "url": "https://ar5iv.labs.arxiv.org/abs/1712.08917"
   },
   {
    "accessed": "2026-09-08",
    "title": "TweetSentBR dataset",
    "url": "https://huggingface.co/datasets/eduagarcia/tweetsentbr_fewshot"
   },
   {
    "accessed": "2026-09-08",
    "title": "Hugging Face datasets-server split sizes for tweetsentbr_fewshot",
    "url": "https://datasets-server.huggingface.co/size?dataset=eduagarcia/tweetsentbr_fewshot"
   }
  ],
  "status": "active",
  "subcategory": "Brazilian Portuguese sentiment",
  "summary": "TweetSentBR evaluates sentiment classification for Brazilian Portuguese tweets.",
  "tags": [
   "sentiment",
   "portuguese",
   "classification"
  ],
  "task_format": "Classify each tweet into one of three labels."
 }
}