English banking intent classification of 13,083 customer-service queries into 77 intents; HELM scores generated labels by exact match.
unassessed
| Category | domain |
|---|---|
| Subcategory | fine-grained English banking intent classification (77 labels) |
| Page status | active |
| Metric | exact match of the intent name (HELM); accuracy in the original paper |
| Direction | higher_is_better |
| Unit | % |
| Dataset size | 13083 |
| Dataset licence | CC-BY-4.0 |
| Publisher | PolyAI Limited |
BANKING77 asks a model to map a short English customer-service utterance to one of 77 fine-grained banking intents (card arrival, failed top-up, reverted transfer, and so on). PolyAI released it as a single-domain contrast to coarser multi-domain sets such as HWU64 and CLINC150. Overlapping intents are deliberate: the model must tell failed top-up from reverted top-up, not just "banking" from "travel".
Single-label text classification. HELM presents the query and generates a label string, then scores quasi-exact / exact match against the canonical intent name. The original paper instead trains a classifier on frozen sentence embeddings and reports accuracy.
No model card in ModelSpec reports this benchmark yet.