The Pile

HELM evaluates language-model perplexity on a test slice of The Pile across its component domains.

unassessed

This page is a discovery lead. Nobody has yet assessed it against the catalogue contract, so it carries no disposition. Absence of evidence here is not evidence of staleness.

Category	generation
Subcategory	language modeling corpus (HELM scenario; see also the lm-evaluation-harness group at pile.md)
Page status	active
Metric	bits_per_byte
Direction	lower_is_better
Unit	bits/byte
Dataset licence	other (per the EleutherAI/pile Hugging Face card); constituent sources keep their own licences
Publisher	EleutherAI

What it measures

The Pile is a large mixed-domain text corpus used for language-model evaluation. HELM scores predictive fit on test documents and supports component subsets such as ArXiv and PhilPapers.

Task format

Autoregressive next-token prediction over text documents.

Models reporting this benchmark

No model card in ModelSpec reports this benchmark yet.

Data

This page as JSON · Edit on GitHub