{
 "body": "\n## What it measures\n\nHospiceReferral asks whether a palliative care note supports referring the\npatient to hospice. HELM's methods text says the context is a palliative care\nconsultation with medical history and clinical status relevant to hospice\neligibility. The model must answer A for yes or B for no. The census spelling\nis shc_gip; HELM's display name is HospiceReferral. This page does not expand\nGIP, because the scenario, schema, and papers never spell those letters out.\n\nThe task is English clinical text, closed labels, one turn. It is an\nend-of-life operations decision, not a general prognosis exam.\n\n## How it is scored\n\nHELM uses joint multiple-choice adaptation with instructions \"Answer A or B.\"\nand scores exact match of the letter. The schema names exact_match on test as\nthe main metric. Quasi-exact and prefix variants are also stored. No human\nhospice-team baseline or class prior was published for the 1000-item private\nset, so a 50 percent chance line is not established.\n\n## Dataset and licence\n\nExtended Data Table 1 reports 1000 instances evaluated, private and new. HELM\nloads medhelm-GIP-dataset_filtered.csv with prompt, context, and label, and\ntags every instance as test. Taxonomy in the paper: Administration and\nWorkflow, scheduling resources and staff. HELM metadata says who is a hospital\nadministrator and when is end-of-care.\n\nThe notes are Stanford Health Care records and are not released. No data\nlicence was found. HELM code is Apache 2.0.\n\n## Who publishes it\n\nHospiceReferral is a new private MedHELM benchmark from Stanford Health Care\nand Stanford CRFM (Bedi, Cui, Fuentes, Unell, Shah and colleagues). Preprint:\narXiv:2505.23802, 26 May 2025. Journal: Nature Medicine, 20 January 2026. The\nHELM scenario entered the public tree with MedHELM V1 on 19 March 2025. The\nleaderboard is at https://crfm.stanford.edu/helm/medhelm/latest/.\n\n## Lineage\n\nThis is not ClinicReferral (shc_sequoia_med), which uses palliative notes to\ndecide Sequoia clinic eligibility, and not ENT-Referral, which is a three-way\nspecialist triage task. It is also not a public hospice-policy quiz. Sibling pages include shc_bmt, shc_cdi, shc_conf, shc_ent, shc_privacy,\nshc_proxy, shc_ptbm, shc_sei, and shc_sequoia (ClinicReferral). There is no\nMedHELM family page.\n\n## Saturation and contamination\n\nSaturation is unknown; no numeric HospiceReferral top score was parsed here.\nContamination risk is low because the authors keep the 1000 notes private.\nPublic web text about hospice criteria could still inflate scores without\nmemorizing this CSV; that effect was not measured.\n\n## How to run it\n\nRun HELM scenario `shc_gip_med` with `data_path` pointing at the private CSV.\nOfficial Stanford rows are in `run_entries_medhelm_private_stanford.conf`.\nAdapter instructions are \"Answer A or B.\" The multiple-choice helper defaults\nto five in-context examples, but only a test split exists, so the run is\neffectively zero-shot. Outside reproduction needs the CSV, which Stanford does\nnot publish. No lm-eval, Inspect, OpenCompass, or BIG-bench twin was confirmed.\n\n## Reading the numbers\n\nA high exact-match score means the model usually matched the gold hospice\nyes or no on these palliative notes. It does not mean the model should make\nhospice decisions: the labels are local, the metric is a single letter, and\nend-of-life criteria include values the note may not state. Compare it with\nClinicReferral and ENT-Referral in the same suite, not with open-ended\nsummarization scores.\n",
 "build": {
  "built_at": "2026-09-09T16:56:50+00:00",
  "commit": "0a599558854c0e238c03a0f0d725239cb28f9d11",
  "eligibility_as_of": "2026-09-09"
 },
 "disposition": {
  "canonical_id": "shc_gip",
  "reasons": [],
  "status": "unassessed",
  "verified_results": []
 },
 "models_covered": [],
 "page": {
  "aliases": [
   "shc_gip_med",
   "HospiceReferral"
  ],
  "category": "domain",
  "contamination": {
   "note": "Listed among the 14 private MedHELM datasets that cannot be shared, which is the authors' stated protection against training inclusion. Those 14 are not all Stanford Health Care records.\n",
   "risk": "low"
  },
  "dataset": {
   "languages": [
    "en"
   ],
   "license": "",
   "modalities": [
    "text"
   ],
   "public_test_set": false,
   "size": 1000,
   "size_note": "Nature Medicine Extended Data Table 1 reports 1000 instances evaluated, private and new. HELM reads medhelm-GIP-dataset_filtered.csv with prompt, context, and label columns and tags every row as test.\n",
   "splits": "test only (HELM TEST_SPLIT); no published train or validation split",
   "url": ""
  },
  "freshness": {
   "researched": "2026-09-08",
   "researched_by": "Grok Build, batch-001 (Codex coordinated)",
   "reviewed": "2026-09-08",
   "reviewed_by": "Grok Build independent review, batch-001"
  },
  "harness": {
   "bigbench": "",
   "helm": "shc_gip_med",
   "inspect_evals": "",
   "lm_eval": "",
   "opencompass": "",
   "other": ""
  },
  "id": "shc_gip",
  "last_updated": "2025-08",
  "leaderboard_url": "https://crfm.stanford.edu/helm/medhelm/latest/",
  "lineage": {
   "family": "",
   "predecessor": "",
   "successors": [],
   "variants": [
    "shc_bmt",
    "shc_cdi",
    "shc_conf",
    "shc_ent",
    "shc_privacy",
    "shc_proxy",
    "shc_ptbm",
    "shc_sei",
    "shc_sequoia"
   ]
  },
  "measures": "HospiceReferral tests whether a model can read an English palliative care consultation note and decide if the patient is eligible for hospice care. HELM's class docstring frames it as a gold-standard referral set used to check hospice referral decisions. The census id is shc_gip; the runnable HELM scenario is shc_gip_med. HELM sources do not expand the letters GIP.\n",
  "metric": {
   "baseline_note": "Main metric is exact_match on A or B. No published class balance or human hospice-coordinator baseline was found for this private set.\n",
   "direction": "higher_is_better",
   "human_baseline": null,
   "max_score": 1.0,
   "name": "exact_match",
   "random_baseline": null,
   "unit": "0-1 scale"
  },
  "name": "HospiceReferral",
  "page_kind": "benchmark",
  "paper": {
   "arxiv": "2505.23802",
   "title": "Holistic evaluation of large language models for medical tasks with MedHELM",
   "url": "https://pmc.ncbi.nlm.nih.gov/articles/PMC13267972/",
   "year": 2026
  },
  "publisher": {
   "authors": [
    "Suhana Bedi",
    "Hejie Cui",
    "Miguel Fuentes",
    "Alyssa Unell",
    "Nigam H. Shah"
   ],
   "org": "Stanford Health Care and Stanford CRFM (MedHELM)",
   "url": "https://crfm.stanford.edu/helm/medhelm/latest/"
  },
  "released": "2025-03",
  "repo_url": "https://github.com/stanford-crfm/helm",
  "saturation": {
   "as_of": "",
   "note": "No dated numeric top exact-match for HospiceReferral was recovered from a parsed leaderboard table.\n",
   "status": "unknown",
   "top_score": null
  },
  "sources": [
   {
    "accessed": "2026-09-08",
    "title": "HELM shc_gip_scenario.py (SHCGIPMedScenario, HospiceReferral)",
    "url": "https://github.com/stanford-crfm/helm/blob/main/src/helm/benchmark/scenarios/shc_gip_scenario.py"
   },
   {
    "accessed": "2026-09-08",
    "title": "HELM medhelm_run_specs.py (shc_gip_med run spec)",
    "url": "https://raw.githubusercontent.com/stanford-crfm/helm/main/src/helm/benchmark/run_specs/medhelm_run_specs.py"
   },
   {
    "accessed": "2026-09-08",
    "title": "HELM run_entries_medhelm_private_stanford.conf",
    "url": "https://raw.githubusercontent.com/stanford-crfm/helm/main/src/helm/benchmark/presentation/run_entries_medhelm_private_stanford.conf"
   },
   {
    "accessed": "2026-09-08",
    "title": "HELM schema_medhelm.yaml (HospiceReferral group)",
    "url": "https://raw.githubusercontent.com/stanford-crfm/helm/main/src/helm/benchmark/static/schema_medhelm.yaml"
   },
   {
    "accessed": "2026-09-08",
    "title": "MedHELM arXiv:2505.23802",
    "url": "https://arxiv.org/abs/2505.23802"
   },
   {
    "accessed": "2026-09-08",
    "title": "MedHELM Nature Medicine author manuscript (PMC13267972)",
    "url": "https://pmc.ncbi.nlm.nih.gov/articles/PMC13267972/"
   },
   {
    "accessed": "2026-09-08",
    "title": "MedHELM documentation",
    "url": "https://crfm-helm.readthedocs.io/en/latest/medhelm/"
   },
   {
    "accessed": "2026-09-08",
    "title": "stanford-crfm/helm Apache License 2.0",
    "url": "https://raw.githubusercontent.com/stanford-crfm/helm/main/LICENSE"
   }
  ],
  "status": "active",
  "subcategory": "hospice eligibility from palliative care notes",
  "summary": "Private Stanford Health Care MedHELM task: from a palliative care note, answer yes or no on whether the patient is eligible for hospice referral.\n",
  "tags": [
   "medical",
   "clinical",
   "classification",
   "hospice",
   "palliative-care",
   "private-dataset",
   "medhelm",
   "stanford-health-care"
  ],
  "task_format": "Joint multiple-choice generation. HELM requires A for yes or B for no, with no extra text, given the CSV question and the palliative note.\n"
 }
}