{
 "body": "\nPart of the [EQ-Bench](eq_bench.md) family.\n\n## What it measures\n\neq_bench_ca is a Catalan translation and cultural adaptation of EQ-Bench, built by the Barcelona Supercomputing Center's Language Technologies Unit from the same dialogue-rating format as the English original. A model reads a short scene of interpersonal conflict and rates the intensity of four named emotions one character is likely feeling. The adapters reworked emotion labels to sidestep Catalan grammatical gender, swapped in Catalan character names, and merged labels that were duplicates of each other once inflection was normalized. The scoring function bundled with the lm-evaluation-harness task is identical to the one the English `eq_bench` v2 task uses, which is the clearest evidence this translation targets EQ-Bench version 2 (171 questions) rather than the smaller version 1 (60 questions) described in the original paper.\n\n## Reading the numbers\n\nA score here measures the same narrow, reference-matching skill as English EQ-Bench v2, applied to Catalan text -- not general Catalan fluency and not the publisher's newer, unrelated EQ-Bench 3/4 roleplay methodology, which has no Catalan port found for this page. Because the item count (167) does not exactly match the 171-question English source, treat this as a distinct, separately-scored artifact rather than a strict translation with guaranteed comparability to English eq_bench numbers.\n",
 "build": {
  "built_at": "2026-09-09T16:56:50+00:00",
  "commit": "0a599558854c0e238c03a0f0d725239cb28f9d11",
  "eligibility_as_of": "2026-09-09"
 },
 "disposition": {
  "canonical_id": "eq_bench_ca",
  "reasons": [],
  "status": "unassessed",
  "verified_results": []
 },
 "models_covered": [],
 "page": {
  "aliases": [
   "EQ-bench_ca"
  ],
  "category": "reasoning",
  "contamination": {
   "note": "Public on Hugging Face since June 2025 under a permissive licence, with no held-out portion or canary string found in the dataset card. No source read for this page documents actual memorization by any model.\n",
   "risk": "medium"
  },
  "dataset": {
   "languages": [
    "ca"
   ],
   "license": "CC BY 4.0",
   "modalities": [
    "text"
   ],
   "public_test_set": true,
   "size": 167,
   "size_note": "167 rows in a single Hugging Face \"test\" split (confirmed via the datasets-server size API). That is close to, but not exactly, the 171-question English v2 set the dataset card names as its source; a handful of items were evidently dropped, merged or failed adaptation during translation. No separate train split exists.\n",
   "splits": "single 167-row test split; no train split",
   "url": "https://huggingface.co/datasets/BSC-LT/EQ-bench_ca"
  },
  "freshness": {
   "researched": "2026-09-08",
   "researched_by": "sonnet-5 agent, batch 5, slice E",
   "reviewed": "",
   "reviewed_by": ""
  },
  "harness": {
   "bigbench": "",
   "helm": "",
   "inspect_evals": "",
   "lm_eval": "eqbench_ca",
   "opencompass": "",
   "other": "Confirmed directly in lm-evaluation-harness: lm_eval/tasks/eq_bench/multilingual/eqbench_ca.yaml sets task: eqbench_ca (metadata version 1.0), reads dataset_path: BSC-LT/EQ-bench_ca, and reuses the shared multilingual utils.py's calculate_score_fullscale scorer -- the same function the English v2 eq_bench task uses."
  },
  "id": "eq_bench_ca",
  "last_updated": "",
  "leaderboard_url": "",
  "lineage": {
   "family": "eq_bench",
   "predecessor": "",
   "successors": [],
   "variants": []
  },
  "measures": "eq_bench_ca shows a model a short dialogue translated and culturally adapted into Catalan, then asks it to rate the intensity (0-10) of four named emotions one character is likely feeling at the end of the scene -- the same task format as English EQ-Bench. The Barcelona Supercomputing Center's Language Technologies Unit produced the adaptation: converting adjectival emotion labels to nominal forms to avoid Catalan grammatical-gender ambiguity, replacing Anglo-Saxon character names with Catalan ones (with correct definite articles), and unifying emotion labels that were equivalent but differently inflected in the English original.\n",
  "metric": {
   "baseline_note": "Uses EQ-Bench v2's unnormalized scoring: small per-emotion differences from the reference are scaled down on an S-shaped curve, larger differences count 1:1, and the total is inverted and rescaled so 0 corresponds to a random answer and 100 to matching the reference exactly. The lm-evaluation-harness `calculate_score_fullscale` function for this task is, line for line, the same scoring function the English `eq_bench` v2 task uses -- confirmed by reading both utils.py files directly -- which is the basis for calling this a v2-based translation rather than v1.\n",
   "direction": "higher_is_better",
   "human_baseline": null,
   "max_score": 100,
   "name": "EQ-Bench score (distance from reference ratings)",
   "random_baseline": 0,
   "unit": "points"
  },
  "name": "EQ-Bench (Catalan)",
  "page_kind": "subset",
  "paper": {
   "arxiv": "2312.06281",
   "title": "EQ-Bench: An Emotional Intelligence Benchmark for Large Language Models",
   "url": "https://arxiv.org/abs/2312.06281",
   "year": 2023
  },
  "publisher": {
   "authors": [],
   "org": "Barcelona Supercomputing Center (BSC), Language Technologies Unit",
   "url": "https://huggingface.co/BSC-LT"
  },
  "released": "2025-06",
  "repo_url": "https://huggingface.co/datasets/BSC-LT/EQ-bench_ca",
  "saturation": {
   "as_of": "",
   "note": "Not established from a source read for this page; no dedicated Catalan EQ-Bench leaderboard was found, and this repository's model cards were not checked for reported eqbench_ca scores.",
   "status": "unknown",
   "top_score": null
  },
  "sources": [
   {
    "accessed": "2026-09-08",
    "title": "BSC-LT/EQ-bench_ca dataset card, Hugging Face",
    "url": "https://huggingface.co/datasets/BSC-LT/EQ-bench_ca"
   },
   {
    "accessed": "2026-09-08",
    "title": "BSC-LT/EQ-bench_ca, Hugging Face Hub API (licence tag, arxiv tag)",
    "url": "https://huggingface.co/api/datasets/BSC-LT/EQ-bench_ca"
   },
   {
    "accessed": "2026-09-08",
    "title": "BSC-LT/EQ-bench_ca row count, Hugging Face datasets-server",
    "url": "https://datasets-server.huggingface.co/size?dataset=BSC-LT/EQ-bench_ca"
   },
   {
    "accessed": "2026-09-08",
    "title": "lm-evaluation-harness: eqbench_ca task config",
    "url": "https://github.com/EleutherAI/lm-evaluation-harness/blob/main/lm_eval/tasks/eq_bench/multilingual/eqbench_ca.yaml"
   },
   {
    "accessed": "2026-09-08",
    "title": "lm-evaluation-harness: eq_bench multilingual utils.py (calculate_score_fullscale)",
    "url": "https://github.com/EleutherAI/lm-evaluation-harness/blob/main/lm_eval/tasks/eq_bench/multilingual/utils.py"
   }
  ],
  "status": "active",
  "subcategory": "emotional and social intelligence (dialogue emotion-intensity prediction), Catalan translation",
  "summary": "Catalan translation and cultural adaptation of EQ-Bench version 2's 167-171 question emotion-intensity rating task, released by the Barcelona Supercomputing Center.",
  "tags": [
   "emotional-intelligence",
   "social-cognition",
   "catalan",
   "subset",
   "translation"
  ],
  "task_format": "Given a Catalan dialogue and four named emotions, output an intensity rating from 0 to 10 for each in a fixed format; scored by the same distance-from-reference formula as EQ-Bench v2 (see the family page). lm-evaluation-harness runs it as task `eqbench_ca`, generating greedily at temperature 0.\n"
 }
}