{
 "body": "\n## What it measures\n\nThe Artificial Analysis Intelligence Index \u2014 this repository's `artificial_analysis_quality_index`\n\u2014 is Artificial Analysis's own composite capability score, built, in the publisher's words, to\ngive \"a single score for tracking progress toward artificial general intelligence across\nmathematics, science, coding, and reasoning.\" It is not one test: version 4.3 blends ten\nindependently-run evaluations across four weighted categories \u2014 Agents (30%), Coding (20%),\nGeneral (30%) and Scientific Reasoning (20%) \u2014 into one weighted number per model. The publisher's\nlive site uses \"Intelligence Index\" as the current name for this score; this page documents it\nunder this repository's id, which predates that branding.\n\n## How it is scored\n\nEach component keeps its own scoring rule: pass@1 on graded tasks such as SciCode and CritPt, an\n\"equality checker\" LLM for open-answer tasks such as Humanity's Last Exam and AA-LCR, and Elo from\npairwise judge comparisons \u2014 a three-judge panel of frontier models \u2014 for open-ended agentic work\nsuch as AA-Briefcase and GDPval-AA v2, the latter anchored so human expert performance scores 1000\non its own internal scale before being frozen and rescaled into the composite. Because most\ncomponents resolve to a percentage-like pass rate, the blended Index is commonly read as roughly a\n0-100 scale, though no stated maximum is published for the composite itself. Testing is zero-shot\nwith standardized prompts, temperature 0 for non-reasoning models and 0.6 for reasoning models, and\nup to 30 automatic retries on API failures.\n\n## Dataset and licence\n\nThere is no single dataset. Version 4.3's ten components range from 70 CritPt research-physics\nproblems to 6,000 AA-Omniscience knowledge questions, mixing Artificial-Analysis-built or\n-licensed private test sets \u2014 AA-Briefcase, AA-Omniscience and AutomationBench-AA are marked\n\"Private Dataset\" on the publisher's evaluations page, specifically to resist contamination \u2014 with\nestablished public benchmarks such as Humanity's Last Exam, SciCode and CritPt. The suite is\ndescribed as \"primarily text-based, English-language\"; GDP.pdf can include document images, so\nthis page records both text and image among its modalities. No single public licence covering the\ncomposite score or methodology text was established from a source read for this page.\n\n## Who publishes it\n\nArtificial Analysis, which describes itself as an independent AI benchmarking company\nheadquartered in San Francisco with an office in Melbourne, designs, runs and publishes every\nofficial Intelligence Index score itself; no third-party or academic co-author was credited on the\nmethodology or announcement pages read for this page.\n\n## Lineage\n\nThe Index has moved through several major versions since v1.0 launched in January 2024: v2.0\n(February 2025), v3.0 (September 2025, adding Terminal-Bench Hard and \ud835\udf0f\u00b2-Bench Telecom), and a\nv4.0 overhaul in January 2026 that dropped MMLU-Pro, LiveCodeBench and AIME 2025 for harder, more\nagentic evaluations such as GDPval-AA and AA-Omniscience. Version 4.3, current at this page's\nresearch date, was announced 7 September 2026 and replaced \ud835\udf0f\u00b3-Banking with AutomationBench-AA\nwhile upgrading Terminal-Bench to v4.0. It belongs to the `artificial_analysis` family alongside\n`artificial_analysis_speed_index`; Artificial Analysis's profession-specific Capability Indices\n(Legal, Healthcare & Medical, Finance & Accounting and others) reuse several of the same\ncomponents under different weights but have no page in this repository yet.\n\n## Saturation and contamination\n\nAs of this page's research date, the top-scoring model on Intelligence Index v4.3 reached a score\nof 53, with two others tied close behind \u2014 well short of an implied 100-point ceiling and\nconsistent with an open, still-separating field on the current, harder suite. Each major version\nreset has itself been partly a response to saturation on the previous suite: v4.0 explicitly\ndropped MMLU-Pro, LiveCodeBench and AIME 2025, benchmarks that had stopped separating frontier\nmodels well. Contamination risk sits at medium: the private components resist public leakage by\ndesign, but the public academic components (Humanity's Last Exam, SciCode, CritPt) carry the\nordinary risk of entering later training data the longer they circulate.\n\n## How to run it\n\nThere is no independent harness a third party can run to reproduce official scores; Artificial\nAnalysis runs every number itself under the protocol documented on its Intelligence Benchmarking\nmethodology page, where prompt templates, answer-extraction regexes and grader-model choices are\npublished. It uses e2b as its sandbox provider for agentic components and its own open-sourced\nharness, Stirrup (github.com/ArtificialAnalysis/Stirrup). Because the component mix, weights and\ngrader models change at every version, scores from different Index versions are not directly\ncomparable, and Artificial Analysis republishes historical model scores under the current version\nrather than preserving old-version numbers on the live site.\n\n## Reading the numbers\n\nA high Intelligence Index score says a model performed well on average across a specific,\ncurrently agent-heavy blend of tasks \u2014 it does not say the model is strong at everything, since a\nmodel that excels at terminal coding but is weak on long-document reasoning can land at a similar\nscore to the reverse profile. Because the weighting shifts with every version toward whatever\nArtificial Analysis judges most representative of frontier capability, compare scores only within\nthe same version, and check the per-evaluation breakdown on the publisher's site if one capability\nmatters more to you than the blended number. Read this figure alongside\n`artificial_analysis_speed_index` and task cost if latency or budget also matter.\n",
 "build": {
  "built_at": "2026-09-09T16:56:50+00:00",
  "commit": "0a599558854c0e238c03a0f0d725239cb28f9d11",
  "eligibility_as_of": "2026-09-09"
 },
 "disposition": {
  "canonical_id": "artificial_analysis_quality_index",
  "reasons": [],
  "status": "unassessed",
  "verified_results": []
 },
 "models_covered": [
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Claude Opus 4.6",
   "model_id": "anthropic/claude-opus-4-6",
   "provider": "anthropic",
   "provider_display": "Anthropic",
   "score": 88.0,
   "source": "lmarena.ai, provider-reports, multimodal-evals, safety-evals, preference-evals, domain-evals, anthropic-system-card-mythos"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "GPT-5.1",
   "model_id": "openai/gpt-5-1",
   "provider": "openai",
   "provider_display": "OpenAI",
   "score": 88.0,
   "source": "lmarena.ai, provider-reports, domain-evals"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "GPT-5.1 Chat",
   "model_id": "openai/gpt-5-1-chat-latest",
   "provider": "openai",
   "provider_display": "OpenAI",
   "score": 88.0,
   "source": "lmarena.ai, provider-reports"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "GPT-5.1 Codex",
   "model_id": "openai/gpt-5-1-codex",
   "provider": "openai",
   "provider_display": "OpenAI",
   "score": 88.0,
   "source": "lmarena.ai, provider-reports"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "GPT-5.1 Codex Max",
   "model_id": "openai/gpt-5-1-codex-max",
   "provider": "openai",
   "provider_display": "OpenAI",
   "score": 88.0,
   "source": "lmarena.ai, provider-reports"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "GPT-5.1 Codex mini",
   "model_id": "openai/gpt-5-1-codex-mini",
   "provider": "openai",
   "provider_display": "OpenAI",
   "score": 88.0,
   "source": "lmarena.ai, provider-reports"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Claude Sonnet 4.5",
   "model_id": "anthropic/claude-sonnet-4-5-20250929",
   "provider": "anthropic",
   "provider_display": "Anthropic",
   "score": 86.0,
   "source": "lmarena.ai, provider-reports, multimodal-evals, safety-evals, preference-evals"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Claude Sonnet 4.5 (latest)",
   "model_id": "anthropic/claude-sonnet-4-5",
   "provider": "anthropic",
   "provider_display": "Anthropic",
   "score": 86.0,
   "source": "lmarena.ai, provider-reports, multimodal-evals, safety-evals, preference-evals"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Claude Sonnet 4",
   "model_id": "anthropic/claude-sonnet-4-20250514",
   "provider": "anthropic",
   "provider_display": "Anthropic",
   "score": 85.0,
   "source": "lmarena.ai, provider-reports, multimodal-evals, safety-evals, preference-evals"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Claude Sonnet 4 (latest)",
   "model_id": "anthropic/claude-sonnet-4-0",
   "provider": "anthropic",
   "provider_display": "Anthropic",
   "score": 85.0,
   "source": "lmarena.ai, provider-reports"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Claude Sonnet 4.6",
   "model_id": "anthropic/claude-sonnet-4-6",
   "provider": "anthropic",
   "provider_display": "Anthropic",
   "score": 85.0,
   "source": "lmarena.ai, provider-reports"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Gemini 2.5 Pro",
   "model_id": "google/gemini-2-5-pro",
   "provider": "google",
   "provider_display": "Google DeepMind",
   "score": 85.0,
   "source": "lmarena.ai, provider-reports, multimodal-evals, safety-evals, preference-evals, domain-evals, llm-stats, intlpull"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Gemini 2.5 Pro Preview 05-06",
   "model_id": "google/gemini-2-5-pro-preview-05-06",
   "provider": "google",
   "provider_display": "Google DeepMind",
   "score": 85.0,
   "source": "lmarena.ai, provider-reports, multimodal-evals, safety-evals, preference-evals, domain-evals"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Gemini 2.5 Pro Preview 06-05",
   "model_id": "google/gemini-2-5-pro-preview-06-05",
   "provider": "google",
   "provider_display": "Google DeepMind",
   "score": 85.0,
   "source": "lmarena.ai, provider-reports, multimodal-evals, safety-evals, preference-evals, domain-evals"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Gemini 2.5 Pro Preview TTS",
   "model_id": "google/gemini-2-5-pro-preview-tts",
   "provider": "google",
   "provider_display": "Google DeepMind",
   "score": 85.0,
   "source": "lmarena.ai, provider-reports, multimodal-evals, safety-evals, preference-evals, domain-evals"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "DeepSeek R1",
   "model_id": "deepseek/deepseek-r1",
   "provider": "deepseek",
   "provider_display": "DeepSeek",
   "score": 84.0,
   "source": "lmarena.ai, provider-reports, preference-evals, open-llm-leaderboard-v2, domain-evals"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "DeepSeek R1 0528",
   "model_id": "deepseek/deepseek-r1-0528",
   "provider": "deepseek",
   "provider_display": "DeepSeek",
   "score": 84.0,
   "source": "lmarena.ai, provider-reports, preference-evals, open-llm-leaderboard-v2, domain-evals"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "DeepSeek R1 0528 NVFP4 v2",
   "model_id": "nvidia/deepseek-r1-0528-nvfp4-v2",
   "provider": "nvidia",
   "provider_display": "NVIDIA",
   "score": 84.0,
   "source": "lmarena.ai, provider-reports, preference-evals, open-llm-leaderboard-v2"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "DeepSeek R1 Distill Llama 70B",
   "model_id": "deepseek/deepseek-r1-distill-llama-70b",
   "provider": "deepseek",
   "provider_display": "DeepSeek",
   "score": 84.0,
   "source": "lmarena.ai, provider-reports, open-llm-leaderboard-v2"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "DeepSeek R1 Distill Llama 8B",
   "model_id": "deepseek/deepseek-r1-distill-llama-8b",
   "provider": "deepseek",
   "provider_display": "DeepSeek",
   "score": 84,
   "source": "lmarena.ai, provider-reports, open-llm-leaderboard-v2"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "DeepSeek R1 Distill Qwen 14B",
   "model_id": "deepseek/deepseek-r1-distill-qwen-14b",
   "provider": "deepseek",
   "provider_display": "DeepSeek",
   "score": 84.0,
   "source": "lmarena.ai, provider-reports, open-llm-leaderboard-v2"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "DeepSeek R1 Distill Qwen 32B",
   "model_id": "deepseek/deepseek-r1-distill-qwen-32b",
   "provider": "deepseek",
   "provider_display": "DeepSeek",
   "score": 84.0,
   "source": "lmarena.ai, provider-reports, open-llm-leaderboard-v2"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "DeepSeek R1 Distill Qwen 7B",
   "model_id": "deepseek/deepseek-r1-distill-qwen-7b",
   "provider": "deepseek",
   "provider_display": "DeepSeek",
   "score": 84,
   "source": "lmarena.ai, provider-reports, open-llm-leaderboard-v2"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "GPT-4.1",
   "model_id": "openai/gpt-4-1",
   "provider": "openai",
   "provider_display": "OpenAI",
   "score": 84.0,
   "source": "lmarena.ai, provider-reports, multimodal-evals, safety-evals, domain-evals preference-evals, llm-stats, intlpull"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "GPT-4o",
   "model_id": "openai/gpt-4o",
   "provider": "openai",
   "provider_display": "OpenAI",
   "score": 82.0,
   "source": "lmarena.ai, provider-reports, multimodal-evals, safety-evals, preference-evals, domain-evals, llm-stats, intlpull"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "GPT-4o (2024-05-13)",
   "model_id": "openai/gpt-4o-2024-05-13",
   "provider": "openai",
   "provider_display": "OpenAI",
   "score": 82.0,
   "source": "lmarena.ai, provider-reports, multimodal-evals, safety-evals, preference-evals, domain-evals, llm-stats, intlpull"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "GPT-4o (2024-08-06)",
   "model_id": "openai/gpt-4o-2024-08-06",
   "provider": "openai",
   "provider_display": "OpenAI",
   "score": 82.0,
   "source": "lmarena.ai, provider-reports, multimodal-evals, safety-evals, preference-evals, domain-evals, llm-stats, intlpull"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "GPT-4o (2024-11-20)",
   "model_id": "openai/gpt-4o-2024-11-20",
   "provider": "openai",
   "provider_display": "OpenAI",
   "score": 82.0,
   "source": "lmarena.ai, provider-reports, multimodal-evals, safety-evals, preference-evals, domain-evals, llm-stats, intlpull"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Grok 3",
   "model_id": "xai/grok-3",
   "provider": "xai",
   "provider_display": "xAI",
   "score": 82.0,
   "source": "lmarena.ai, provider-reports, domain-evals"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Grok 3 Fast",
   "model_id": "xai/grok-3-fast",
   "provider": "xai",
   "provider_display": "xAI",
   "score": 82.0,
   "source": "lmarena.ai, provider-reports"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Grok 3 Fast Latest",
   "model_id": "xai/grok-3-fast-latest",
   "provider": "xai",
   "provider_display": "xAI",
   "score": 82.0,
   "source": "lmarena.ai, provider-reports"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Grok 3 Latest",
   "model_id": "xai/grok-3-latest",
   "provider": "xai",
   "provider_display": "xAI",
   "score": 82.0,
   "source": "lmarena.ai, provider-reports"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Qwen3 235B-A22B",
   "model_id": "qwen/qwen3-235b-a22b",
   "provider": "qwen",
   "provider_display": "Alibaba / Qwen Team",
   "score": 82.0,
   "source": "lmarena.ai, provider-reports, preference-evals, llm-stats, domain-evals"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "DeepSeek V3",
   "model_id": "deepseek/deepseek-v3",
   "provider": "deepseek",
   "provider_display": "DeepSeek",
   "score": 80.0,
   "source": "lmarena.ai, provider-reports, safety-evals, preference-evals, domain-evals open-llm-leaderboard-v2, llm-stats"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "DeepSeek V3 0324",
   "model_id": "deepseek/deepseek-v3-0324",
   "provider": "deepseek",
   "provider_display": "DeepSeek",
   "score": 80.0,
   "source": "lmarena.ai, provider-reports, safety-evals, preference-evals, open-llm-leaderboard-v2, llm-stats"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "DeepSeek V3.1",
   "model_id": "deepseek/deepseek-v3-1",
   "provider": "deepseek",
   "provider_display": "DeepSeek",
   "score": 80.0,
   "source": "lmarena.ai, provider-reports, safety-evals, preference-evals, open-llm-leaderboard-v2, llm-stats"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "DeepSeek V3.2",
   "model_id": "deepseek/deepseek-v3-2",
   "provider": "deepseek",
   "provider_display": "DeepSeek",
   "score": 80.0,
   "source": "lmarena.ai, provider-reports, safety-evals, preference-evals, open-llm-leaderboard-v2, llm-stats"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "DeepSeek V3.2 Exp",
   "model_id": "deepseek/deepseek-v3-2-exp",
   "provider": "deepseek",
   "provider_display": "DeepSeek",
   "score": 80.0,
   "source": "lmarena.ai, provider-reports, safety-evals, preference-evals, open-llm-leaderboard-v2, llm-stats"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Gemini 2.5 Flash",
   "model_id": "google/gemini-2-5-flash",
   "provider": "google",
   "provider_display": "Google DeepMind",
   "score": 80.0,
   "source": "lmarena.ai, provider-reports, llm-stats, intlpull"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Gemini 2.5 Flash Image",
   "model_id": "google/gemini-2-5-flash-image",
   "provider": "google",
   "provider_display": "Google DeepMind",
   "score": 80.0,
   "source": "lmarena.ai, provider-reports"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Gemini 2.5 Flash Image (Preview)",
   "model_id": "google/gemini-2-5-flash-image-preview",
   "provider": "google",
   "provider_display": "Google DeepMind",
   "score": 80.0,
   "source": "lmarena.ai, provider-reports"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Gemini 2.5 Flash Lite",
   "model_id": "google/gemini-2-5-flash-lite",
   "provider": "google",
   "provider_display": "Google DeepMind",
   "score": 80.0,
   "source": "lmarena.ai, provider-reports"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Gemini 2.5 Flash Lite Preview 06-17",
   "model_id": "google/gemini-2-5-flash-lite-preview-06-17",
   "provider": "google",
   "provider_display": "Google DeepMind",
   "score": 80.0,
   "source": "lmarena.ai, provider-reports"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Gemini 2.5 Flash Lite Preview 09-25",
   "model_id": "google/gemini-2-5-flash-lite-preview-09-2025",
   "provider": "google",
   "provider_display": "Google DeepMind",
   "score": 80.0,
   "source": "lmarena.ai, provider-reports"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Gemini 2.5 Flash Preview 04-17",
   "model_id": "google/gemini-2-5-flash-preview-04-17",
   "provider": "google",
   "provider_display": "Google DeepMind",
   "score": 80.0,
   "source": "lmarena.ai, provider-reports"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Gemini 2.5 Flash Preview 05-20",
   "model_id": "google/gemini-2-5-flash-preview-05-20",
   "provider": "google",
   "provider_display": "Google DeepMind",
   "score": 80.0,
   "source": "lmarena.ai, provider-reports"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Gemini 2.5 Flash Preview 09-25",
   "model_id": "google/gemini-2-5-flash-preview-09-2025",
   "provider": "google",
   "provider_display": "Google DeepMind",
   "score": 80.0,
   "source": "lmarena.ai, provider-reports"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Gemini 2.5 Flash Preview TTS",
   "model_id": "google/gemini-2-5-flash-preview-tts",
   "provider": "google",
   "provider_display": "Google DeepMind",
   "score": 80.0,
   "source": "lmarena.ai, provider-reports"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Llama 3.1 405B",
   "model_id": "meta/llama-3-1-405b",
   "provider": "meta",
   "provider_display": "Meta",
   "score": 80.0,
   "source": "lmarena.ai, provider-reports, llm-stats"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Llama 3.1 405B FP8",
   "model_id": "meta/llama-3-1-405b-fp8",
   "provider": "meta",
   "provider_display": "Meta",
   "score": 80.0,
   "source": "lmarena.ai, provider-reports"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Llama 3.1 405B Instruct",
   "model_id": "meta/llama-3-1-405b-instruct",
   "provider": "meta",
   "provider_display": "Meta",
   "score": 80.0,
   "source": "lmarena.ai, provider-reports, llm-stats, domain-evals"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Llama 3.1 405B Instruct FP8",
   "model_id": "meta/llama-3-1-405b-instruct-fp8",
   "provider": "meta",
   "provider_display": "Meta",
   "score": 80.0,
   "source": "lmarena.ai, provider-reports, llm-stats"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "GPT-4 Turbo",
   "model_id": "openai/gpt-4-turbo",
   "provider": "openai",
   "provider_display": "OpenAI",
   "score": 79.0,
   "source": "lmarena.ai, provider-reports, llm-stats, intlpull, multimodal-evals, safety-evals"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Command A",
   "model_id": "cohere/command-a-03-2025",
   "provider": "cohere",
   "provider_display": "Cohere",
   "score": 78,
   "source": "lmarena.ai, provider-reports"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Command A Reasoning",
   "model_id": "cohere/command-a-reasoning-08-2025",
   "provider": "cohere",
   "provider_display": "Cohere",
   "score": 78,
   "source": "lmarena.ai, provider-reports"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Gemini 1.5 Pro",
   "model_id": "google/gemini-1-5-pro",
   "provider": "google",
   "provider_display": "Google DeepMind",
   "score": 78.0,
   "source": "lmarena.ai, provider-reports, multimodal-evals, llm-stats, intlpull, safety-evals"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Mistral Large (latest)",
   "model_id": "mistral/mistral-large-latest",
   "provider": "mistral",
   "provider_display": "Mistral AI",
   "score": 78.0,
   "source": "lmarena.ai, provider-reports, multimodal-evals, safety-evals, preference-evals, open-llm-leaderboard-v2, llm-stats"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Mistral Large 2.1",
   "model_id": "mistral/mistral-large-2411",
   "provider": "mistral",
   "provider_display": "Mistral AI",
   "score": 78.0,
   "source": "lmarena.ai, provider-reports, safety-evals, preference-evals,, domain-evals open-llm-leaderboard-v2"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Mistral Large 3",
   "model_id": "mistral/mistral-large-2512",
   "provider": "mistral",
   "provider_display": "Mistral AI",
   "score": 78.0,
   "source": "lmarena.ai, provider-reports, multimodal-evals, safety-evals, preference-evals, open-llm-leaderboard-v2"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Gemini 2.0 Flash",
   "model_id": "google/gemini-2-0-flash",
   "provider": "google",
   "provider_display": "Google DeepMind",
   "score": 77.0,
   "source": "lmarena.ai, provider-reports, multimodal-evals, llm-stats, intlpull, domain-evals, safety-evals"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Grok 2",
   "model_id": "xai/grok-2",
   "provider": "xai",
   "provider_display": "xAI",
   "score": 76,
   "source": "lmarena.ai, provider-reports, multimodal-evals"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Grok 2 (1212)",
   "model_id": "xai/grok-2-1212",
   "provider": "xai",
   "provider_display": "xAI",
   "score": 76,
   "source": "lmarena.ai, provider-reports"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Grok 2 Latest",
   "model_id": "xai/grok-2-latest",
   "provider": "xai",
   "provider_display": "xAI",
   "score": 76,
   "source": "lmarena.ai, provider-reports, multimodal-evals"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Llama 3.3 70B Instruct NVFP4",
   "model_id": "nvidia/llama-3-3-70b-instruct-nvfp4",
   "provider": "nvidia",
   "provider_display": "NVIDIA",
   "score": 76.0,
   "source": "lmarena.ai, provider-reports, safety-evals, preference-evals, open-llm-leaderboard-v2, llm-stats"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Llama-3.3-70B-Instruct",
   "model_id": "meta/llama-3-3-70b-instruct",
   "provider": "meta",
   "provider_display": "Meta",
   "score": 76.0,
   "source": "lmarena.ai, provider-reports, safety-evals, preference-evals, open-llm-leaderboard-v2, llm-stats"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Qwen3 32B",
   "model_id": "qwen/qwen3-32b",
   "provider": "qwen",
   "provider_display": "Alibaba / Qwen Team",
   "score": 76.0,
   "source": "lmarena.ai, provider-reports, preference-evals, open-llm-leaderboard-v2"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Qwen3 32B AWQ",
   "model_id": "qwen/qwen3-32b-awq",
   "provider": "qwen",
   "provider_display": "Alibaba / Qwen Team",
   "score": 76.0,
   "source": "lmarena.ai, provider-reports, preference-evals, open-llm-leaderboard-v2"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Qwen3 32B NVFP4",
   "model_id": "nvidia/qwen3-32b-nvfp4",
   "provider": "nvidia",
   "provider_display": "NVIDIA",
   "score": 76.0,
   "source": "lmarena.ai, provider-reports, preference-evals, open-llm-leaderboard-v2"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Llama 3.1 70B",
   "model_id": "meta/llama-3-1-70b",
   "provider": "meta",
   "provider_display": "Meta",
   "score": 75.0,
   "source": "lmarena.ai, provider-reports, safety-evals, domain-evals, open-llm-leaderboard-v2"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Llama 3.1 70B Instruct",
   "model_id": "meta/llama-3-1-70b-instruct",
   "provider": "meta",
   "provider_display": "Meta",
   "score": 75.0,
   "source": "lmarena.ai, provider-reports, safety-evals, domain-evals, open-llm-leaderboard-v2"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "GPT-4.1 mini",
   "model_id": "openai/gpt-4-1-mini",
   "provider": "openai",
   "provider_display": "OpenAI",
   "score": 74.0,
   "source": "lmarena.ai, provider-reports, multimodal-evals, safety-evals, preference-evals, llm-stats, intlpull"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Qwen3 30B A3B Instruct 2507",
   "model_id": "qwen/qwen3-30b-a3b-instruct-2507",
   "provider": "qwen",
   "provider_display": "Alibaba / Qwen Team",
   "score": 74.0,
   "source": "lmarena.ai, provider-reports, open-llm-leaderboard-v2"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Qwen3 30B A3B NVFP4",
   "model_id": "nvidia/qwen3-30b-a3b-nvfp4",
   "provider": "nvidia",
   "provider_display": "NVIDIA",
   "score": 74.0,
   "source": "lmarena.ai, provider-reports, open-llm-leaderboard-v2"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Qwen3 30B-A3B",
   "model_id": "qwen/qwen3-30b-a3b",
   "provider": "qwen",
   "provider_display": "Alibaba / Qwen Team",
   "score": 74.0,
   "source": "lmarena.ai, provider-reports, open-llm-leaderboard-v2, domain-evals"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Command R+",
   "model_id": "cohere/command-r-plus-08-2024",
   "provider": "cohere",
   "provider_display": "Cohere",
   "score": 73.0,
   "source": "lmarena.ai, provider-reports, safety-evals, preference-evals, open-llm-leaderboard-v2, llm-stats"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Gemma 4 31B",
   "model_id": "google/gemma-4-31b",
   "provider": "google",
   "provider_display": "Google DeepMind",
   "score": 73.0,
   "source": "lmarena.ai, provider-reports, multimodal-evals, open-llm-leaderboard-v2"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "gemma 4 31B it",
   "model_id": "google/gemma-4-31b-it",
   "provider": "google",
   "provider_display": "Google DeepMind",
   "score": 73.0,
   "source": "lmarena.ai, provider-reports, multimodal-evals, open-llm-leaderboard-v2"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "gemma 4 31B it GGUF",
   "model_id": "unsloth/gemma-4-31b-it-gguf",
   "provider": "unsloth",
   "provider_display": "Unsloth",
   "score": 73.0,
   "source": "lmarena.ai, provider-reports, multimodal-evals, open-llm-leaderboard-v2"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Gemma 4 31B IT NVFP4",
   "model_id": "nvidia/gemma-4-31b-it-nvfp4",
   "provider": "nvidia",
   "provider_display": "NVIDIA",
   "score": 73.0,
   "source": "lmarena.ai, provider-reports, multimodal-evals, open-llm-leaderboard-v2"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "GPT-4o mini",
   "model_id": "openai/gpt-4o-mini",
   "provider": "openai",
   "provider_display": "OpenAI",
   "score": 72.0,
   "source": "lmarena.ai, provider-reports, multimodal-evals, safety-evals, preference-evals, domain-evals, llm-stats, intlpull"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Grok 3 Mini",
   "model_id": "xai/grok-3-mini",
   "provider": "xai",
   "provider_display": "xAI",
   "score": 72.0,
   "source": "lmarena.ai, provider-reports"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Grok 3 Mini Fast",
   "model_id": "xai/grok-3-mini-fast",
   "provider": "xai",
   "provider_display": "xAI",
   "score": 72.0,
   "source": "lmarena.ai, provider-reports"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Grok 3 Mini Fast Latest",
   "model_id": "xai/grok-3-mini-fast-latest",
   "provider": "xai",
   "provider_display": "xAI",
   "score": 72.0,
   "source": "lmarena.ai, provider-reports"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Grok 3 Mini Latest",
   "model_id": "xai/grok-3-mini-latest",
   "provider": "xai",
   "provider_display": "xAI",
   "score": 72.0,
   "source": "lmarena.ai, provider-reports"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Gemini 1.5 Flash",
   "model_id": "google/gemini-1-5-flash",
   "provider": "google",
   "provider_display": "Google DeepMind",
   "score": 71.0,
   "source": "lmarena.ai, provider-reports, llm-stats, intlpull, multimodal-evals, safety-evals"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Gemini 1.5 Flash-8B",
   "model_id": "google/gemini-1-5-flash-8b",
   "provider": "google",
   "provider_display": "Google DeepMind",
   "score": 71.0,
   "source": "lmarena.ai, provider-reports, llm-stats, intlpull, multimodal-evals, safety-evals"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Gemma 3 27B",
   "model_id": "google/gemma-3-27b-it",
   "provider": "google",
   "provider_display": "Google DeepMind",
   "score": 70.0,
   "source": "lmarena.ai, provider-reports, multimodal-evals, open-llm-leaderboard-v2,, domain-evals llm-stats"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Qwen3 14B",
   "model_id": "qwen/qwen3-14b",
   "provider": "qwen",
   "provider_display": "Alibaba / Qwen Team",
   "score": 70,
   "source": "lmarena.ai, provider-reports, open-llm-leaderboard-v2"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Qwen3 14B AWQ",
   "model_id": "qwen/qwen3-14b-awq",
   "provider": "qwen",
   "provider_display": "Alibaba / Qwen Team",
   "score": 70,
   "source": "lmarena.ai, provider-reports, open-llm-leaderboard-v2"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Qwen3 14B NVFP4",
   "model_id": "nvidia/qwen3-14b-nvfp4",
   "provider": "nvidia",
   "provider_display": "NVIDIA",
   "score": 70,
   "source": "lmarena.ai, provider-reports, open-llm-leaderboard-v2"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Gemini 2.0 Flash Lite",
   "model_id": "google/gemini-2-0-flash-lite",
   "provider": "google",
   "provider_display": "Google DeepMind",
   "score": 68.0,
   "source": "lmarena.ai, provider-reports, multimodal-evals, safety-evals"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "gemma 2 27B it",
   "model_id": "google/gemma-2-27b-it",
   "provider": "google",
   "provider_display": "Google DeepMind",
   "score": 68.0,
   "source": "lmarena.ai, provider-reports, open-llm-leaderboard-v2, llm-stats"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Mistral Small (latest)",
   "model_id": "mistral/mistral-small-latest",
   "provider": "mistral",
   "provider_display": "Mistral AI",
   "score": 68.0,
   "source": "lmarena.ai"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Mistral Small 24B Instruct 2501",
   "model_id": "mistral/mistral-small-24b-instruct-2501",
   "provider": "mistral",
   "provider_display": "Mistral AI",
   "score": 68.0,
   "source": "lmarena.ai"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Mistral Small 3.1 24B Instruct 2503",
   "model_id": "mistral/mistral-small-3-1-24b-instruct-2503",
   "provider": "mistral",
   "provider_display": "Mistral AI",
   "score": 68.0,
   "source": "lmarena.ai"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Mistral Small 3.2",
   "model_id": "mistral/mistral-small-2506",
   "provider": "mistral",
   "provider_display": "Mistral AI",
   "score": 68.0,
   "source": "lmarena.ai"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Mistral Small 3.2 24B Instruct 2506",
   "model_id": "mistral/mistral-small-3-2-24b-instruct-2506",
   "provider": "mistral",
   "provider_display": "Mistral AI",
   "score": 68.0,
   "source": "lmarena.ai"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Mistral Small 4",
   "model_id": "mistral/mistral-small-2603",
   "provider": "mistral",
   "provider_display": "Mistral AI",
   "score": 68.0,
   "source": "lmarena.ai"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Mistral Small 4 119B 2603",
   "model_id": "mistral/mistral-small-4-119b-2603",
   "provider": "mistral",
   "provider_display": "Mistral AI",
   "score": 68.0,
   "source": "lmarena.ai"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "phi 4",
   "model_id": "microsoft/phi-4",
   "provider": "microsoft",
   "provider_display": "Microsoft",
   "score": 68.0,
   "source": "lmarena.ai, provider-reports, preference-evals, open-llm-leaderboard-v2,, domain-evals llm-stats"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Phi 4 mini instruct",
   "model_id": "microsoft/phi-4-mini-instruct",
   "provider": "microsoft",
   "provider_display": "Microsoft",
   "score": 68.0,
   "source": "lmarena.ai, provider-reports, preference-evals, llm-stats, open-llm-leaderboard-v2"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Qwen3 8B",
   "model_id": "qwen/qwen3-8b",
   "provider": "qwen",
   "provider_display": "Alibaba / Qwen Team",
   "score": 66,
   "source": "lmarena.ai, provider-reports, open-llm-leaderboard-v2"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Qwen3 8B AWQ",
   "model_id": "qwen/qwen3-8b-awq",
   "provider": "qwen",
   "provider_display": "Alibaba / Qwen Team",
   "score": 66,
   "source": "lmarena.ai, provider-reports, open-llm-leaderboard-v2"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Qwen3 8B Base",
   "model_id": "qwen/qwen3-8b-base",
   "provider": "qwen",
   "provider_display": "Alibaba / Qwen Team",
   "score": 66,
   "source": "lmarena.ai, provider-reports, open-llm-leaderboard-v2"
  },
  {
   "as_of": null,
   "attribution": "unverified-legacy",
   "display_name": "Skywork Reward V2 Qwen3 8B",
   "model_id": "skywork/skywork-reward-v2-qwen3-8b",
   "provider": "skywork",
   "provider_display": "Skywork",
   "score": 66,
   "source": null
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Gemma 3 12B",
   "model_id": "google/gemma-3-12b-it",
   "provider": "google",
   "provider_display": "Google DeepMind",
   "score": 65.0,
   "source": "lmarena.ai, provider-reports, open-llm-leaderboard-v2, llm-stats, multimodal-evals"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "GPT-4.1 nano",
   "model_id": "openai/gpt-4-1-nano",
   "provider": "openai",
   "provider_display": "OpenAI",
   "score": 65.0,
   "source": "lmarena.ai, provider-reports, multimodal-evals, safety-evals, preference-evals, llm-stats, intlpull"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Mistral Nemo",
   "model_id": "mistral/mistral-nemo",
   "provider": "mistral",
   "provider_display": "Mistral AI",
   "score": 65,
   "source": "open-llm-leaderboard-v2"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Mistral Nemo Base 2407",
   "model_id": "mistral/mistral-nemo-base-2407",
   "provider": "mistral",
   "provider_display": "Mistral AI",
   "score": 65,
   "source": "open-llm-leaderboard-v2"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Mistral Nemo Instruct 2407",
   "model_id": "mistral/mistral-nemo-instruct-2407",
   "provider": "mistral",
   "provider_display": "Mistral AI",
   "score": 65,
   "source": "open-llm-leaderboard-v2"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Command R",
   "model_id": "cohere/command-r-08-2024",
   "provider": "cohere",
   "provider_display": "Cohere",
   "score": 64,
   "source": "lmarena.ai, provider-reports, open-llm-leaderboard-v2"
  },
  {
   "as_of": null,
   "attribution": "unverified-legacy",
   "display_name": "Arctic LSTM Speculator Llama 3.1 8B Instruct",
   "model_id": "snowflake/arctic-lstm-speculator-llama-3-1-8b-instruct",
   "provider": "snowflake",
   "provider_display": "Snowflake",
   "score": 62,
   "source": null
  },
  {
   "as_of": "2025-03",
   "attribution": "unverified-legacy",
   "display_name": "Hermes 3 Llama 3.1 8B",
   "model_id": "nous-research/hermes-3-llama-3-1-8b",
   "provider": "nous-research",
   "provider_display": "Nous Research",
   "score": 62.0,
   "source": "open-llm-leaderboard-v2"
  },
  {
   "as_of": "2025-03",
   "attribution": "unverified-legacy",
   "display_name": "Hermes 3 Llama 3.1 8B GGUF",
   "model_id": "nous-research/hermes-3-llama-3-1-8b-gguf",
   "provider": "nous-research",
   "provider_display": "Nous Research",
   "score": 62.0,
   "source": "open-llm-leaderboard-v2"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Llama 3.1 8B",
   "model_id": "meta/llama-3-1-8b",
   "provider": "meta",
   "provider_display": "Meta",
   "score": 62,
   "source": "lmarena.ai, provider-reports, safety-evals, open-llm-leaderboard-v2"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Llama 3.1 8B Instruct",
   "model_id": "meta/llama-3-1-8b-instruct",
   "provider": "meta",
   "provider_display": "Meta",
   "score": 62,
   "source": "lmarena.ai, provider-reports, safety-evals, open-llm-leaderboard-v2"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Llama 3.1 8B Instruct",
   "model_id": "unsloth/llama-3-1-8b-instruct",
   "provider": "unsloth",
   "provider_display": "Unsloth",
   "score": 62,
   "source": "lmarena.ai, provider-reports, safety-evals, open-llm-leaderboard-v2"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Llama 3.1 8B Instruct FP8",
   "model_id": "nvidia/llama-3-1-8b-instruct-fp8",
   "provider": "nvidia",
   "provider_display": "NVIDIA",
   "score": 62,
   "source": "lmarena.ai, provider-reports, safety-evals, open-llm-leaderboard-v2"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Llama 3.1 8B Instruct NVFP4",
   "model_id": "nvidia/llama-3-1-8b-instruct-nvfp4",
   "provider": "nvidia",
   "provider_display": "NVIDIA",
   "score": 62,
   "source": "lmarena.ai, provider-reports, safety-evals, open-llm-leaderboard-v2"
  },
  {
   "as_of": null,
   "attribution": "unverified-legacy",
   "display_name": "Meta Llama 3.1 8B",
   "model_id": "nous-research/meta-llama-3-1-8b",
   "provider": "nous-research",
   "provider_display": "Nous Research",
   "score": 62,
   "source": null
  },
  {
   "as_of": null,
   "attribution": "unverified-legacy",
   "display_name": "Meta Llama 3.1 8B Instruct",
   "model_id": "nous-research/meta-llama-3-1-8b-instruct",
   "provider": "nous-research",
   "provider_display": "Nous Research",
   "score": 62,
   "source": null
  },
  {
   "as_of": null,
   "attribution": "unverified-legacy",
   "display_name": "Meta Llama 3.1 8B Instruct",
   "model_id": "unsloth/meta-llama-3-1-8b-instruct",
   "provider": "unsloth",
   "provider_display": "Unsloth",
   "score": 62,
   "source": null
  },
  {
   "as_of": null,
   "attribution": "unverified-legacy",
   "display_name": "Meta Llama 3.1 8B Instruct bnb 4bit",
   "model_id": "unsloth/meta-llama-3-1-8b-instruct-bnb-4bit",
   "provider": "unsloth",
   "provider_display": "Unsloth",
   "score": 62,
   "source": null
  },
  {
   "as_of": null,
   "attribution": "unverified-legacy",
   "display_name": "Skywork Reward Llama 3.1 8B v0.2",
   "model_id": "skywork/skywork-reward-llama-3-1-8b-v0-2",
   "provider": "skywork",
   "provider_display": "Skywork",
   "score": 62,
   "source": null
  },
  {
   "as_of": null,
   "attribution": "unverified-legacy",
   "display_name": "Skywork Reward V2 Llama 3.1 8B",
   "model_id": "skywork/skywork-reward-v2-llama-3-1-8b",
   "provider": "skywork",
   "provider_display": "Skywork",
   "score": 62,
   "source": null
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "gemma 2 9B",
   "model_id": "google/gemma-2-9b",
   "provider": "google",
   "provider_display": "Google DeepMind",
   "score": 60.0,
   "source": "lmarena.ai, provider-reports, open-llm-leaderboard-v2, llm-stats"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "gemma 2 9B it",
   "model_id": "google/gemma-2-9b-it",
   "provider": "google",
   "provider_display": "Google DeepMind",
   "score": 60.0,
   "source": "lmarena.ai, provider-reports, open-llm-leaderboard-v2, llm-stats"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "GPT-3.5-turbo",
   "model_id": "openai/gpt-3-5-turbo",
   "provider": "openai",
   "provider_display": "OpenAI",
   "score": 60.0,
   "source": "lmarena.ai, provider-reports, llm-stats"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Phi 3.5 mini instruct",
   "model_id": "microsoft/phi-3-5-mini-instruct",
   "provider": "microsoft",
   "provider_display": "Microsoft",
   "score": 58.0,
   "source": "open-llm-leaderboard-v2, llm-stats"
  },
  {
   "as_of": "2026-04",
   "attribution": "unverified-legacy",
   "display_name": "Muse Spark",
   "model_id": "meta/muse-spark",
   "provider": "meta",
   "provider_display": "Meta",
   "score": 52.0,
   "source": "meta-blog, officechai, artificial-analysis"
  }
 ],
 "page": {
  "aliases": [
   "Artificial Analysis Quality Index",
   "AA Intelligence Index",
   "AAII"
  ],
  "category": "composite",
  "contamination": {
   "note": "The private components (AA-Briefcase, AA-Omniscience, AutomationBench-AA) resist public leakage by design, but the public academic components (Humanity's Last Exam, SciCode, CritPt) carry the ordinary risk of entering later training data the longer they circulate.\n",
   "risk": "medium"
  },
  "dataset": {
   "languages": [
    "en"
   ],
   "license": "",
   "modalities": [
    "text",
    "image"
   ],
   "public_test_set": null,
   "size": null,
   "size_note": "Ten component evaluations as of v4.3 (AA-Briefcase, GDPval-AA v2, AutomationBench-AA, Terminal-Bench v4.0, SciCode, AA-Omniscience, GDP.pdf, AA-LCR v1.1, Humanity's Last Exam, CritPt), individually ranging from 70 CritPt problems to 6,000 AA-Omniscience questions.\n",
   "splits": "",
   "url": "https://artificialanalysis.ai/evaluations/artificial-analysis-intelligence-index"
  },
  "freshness": {
   "researched": "2026-09-08",
   "researched_by": "sonnet-5 agent, batch 1b, slice J",
   "reviewed": "",
   "reviewed_by": ""
  },
  "harness": {
   "bigbench": "",
   "helm": "",
   "inspect_evals": "",
   "lm_eval": "",
   "opencompass": "",
   "other": "No independent harness a third party runs; Artificial Analysis runs the suite itself under the protocol documented on its Intelligence Benchmarking methodology page (prompt templates, answer-extraction regexes and grader-model choices are published there). It uses e2b as its sandbox provider for agentic components and its own open-sourced harness, Stirrup (github.com/ArtificialAnalysis/Stirrup), for tool-using evaluations.\n"
  },
  "id": "artificial_analysis_quality_index",
  "last_updated": "2026-09",
  "leaderboard_url": "https://artificialanalysis.ai/evaluations/artificial-analysis-intelligence-index",
  "lineage": {
   "family": "artificial_analysis",
   "predecessor": "",
   "successors": [],
   "variants": []
  },
  "measures": "The Artificial Analysis Intelligence Index \u2014 this repository's artificial_analysis_quality_index \u2014 is Artificial Analysis's composite capability score, built, in the publisher's own words, to give \"a single score for tracking progress toward artificial general intelligence across mathematics, science, coding, and reasoning.\" It is not one test: version 4.3 blends ten independently-run evaluations across four weighted categories \u2014 Agents (30%: AA-Briefcase, GDPval-AA v2, AutomationBench-AA), Coding (20%: Terminal-Bench v4.0, SciCode), General (30%: AA-Omniscience, GDP.pdf, AA-LCR v1.1) and Scientific Reasoning (20%: Humanity's Last Exam, CritPt) \u2014 into one weighted number per model. The publisher's live site uses \"Intelligence Index\" as the current name for this score; this page documents it under this repository's id, which predates that branding.\n",
  "metric": {
   "baseline_note": "No single random baseline: each of the ten component evaluations keeps its own scoring rule, and most report pass@1-style percentages, so the composite is commonly discussed as if on a 0-100 scale, though Artificial Analysis does not publish a stated maximum for the composite itself. GDPval-AA v2 separately anchors human expert performance at 1000 on its own internal Elo scale before being frozen and rescaled into the composite \u2014 a component-level anchor, not a stated baseline for the Index as a whole.\n",
   "direction": "higher_is_better",
   "human_baseline": null,
   "max_score": null,
   "name": "Intelligence Index score",
   "random_baseline": null,
   "unit": "points"
  },
  "name": "Artificial Analysis Intelligence Index",
  "page_kind": "benchmark",
  "paper": {
   "arxiv": "",
   "title": "",
   "url": "",
   "year": null
  },
  "publisher": {
   "authors": [],
   "org": "Artificial Analysis",
   "url": "https://artificialanalysis.ai"
  },
  "released": "2024-01",
  "repo_url": "",
  "saturation": {
   "as_of": "2026-09",
   "note": "As of this page's research date, the top-scoring model on Intelligence Index v4.3 reached a score of 53, with two others tied close behind \u2014 well short of an implied 100-point ceiling and consistent with an open, still-separating field on the current, harder suite.\n",
   "status": "open",
   "top_score": 53
  },
  "sources": [
   {
    "accessed": "2026-09-08",
    "title": "Artificial Analysis Intelligence Benchmarking Methodology",
    "url": "https://artificialanalysis.ai/methodology/intelligence-benchmarking"
   },
   {
    "accessed": "2026-09-08",
    "title": "Artificial Analysis Intelligence Index \u2014 evaluation leaderboard",
    "url": "https://artificialanalysis.ai/evaluations/artificial-analysis-intelligence-index"
   },
   {
    "accessed": "2026-09-08",
    "title": "Announcing the Artificial Analysis Intelligence Index v4.3",
    "url": "https://artificialanalysis.ai/articles/artificial-analysis-intelligence-index-v4-3"
   },
   {
    "accessed": "2026-09-08",
    "title": "Artificial Analysis (homepage)",
    "url": "https://artificialanalysis.ai/"
   }
  ],
  "status": "active",
  "subcategory": "composite capability / intelligence score",
  "summary": "Artificial Analysis's own composite capability score, blending ten independently-run evaluations across agents, coding, general knowledge and scientific reasoning into one weighted number.",
  "tags": [
   "composite",
   "agentic",
   "reasoning",
   "artificial-analysis"
  ],
  "task_format": "A weighted blend of ten independently-scored evaluation tasks spanning agentic file/tool tasks, terminal-based coding tasks, open-answer knowledge and reasoning questions, and long-document and physics-reasoning problems; each component evaluation keeps its own response format and scoring rule before being combined.\n"
 }
}