OpenCompass QASPER variant that keeps extractive-span questions and feeds the paper text from the first gold evidence offset onward.
unassessed
| Category | long-context |
|---|---|
| Subcategory | OpenCompass QASPER extractive QA with the paper cut at first evidence |
| Page status | unknown |
| Metric | TriviaQA token F1 (TriviaQAEvaluator) |
| Direction | higher_is_better |
| Unit | % |
| Dataset licence | CC-BY-4.0 |
| Publisher | Allen Institute for AI (QASPER data); OpenCompass (cut wrap) |
qaspercut is not a second copy of [qasper](qasper.md). It uses the same QASPER validation JSON, but it drops every question that has no extractive span and it truncates each paper. The loader finds the earliest gold evidence string in the concatenated full text, then passes only the suffix from that offset as the prompt context. The model still has to read a long remainder of the paper, but it is handed a cut that starts at (or near) a human-highlighted clue instead of the title page.
Zero-shot generation on the QASPER dev split. Prompt is evidence suffix, then "Answer these questions: Q: {question}? A:". Gold is the list of extractive spans. Evaluator is OpenCompass TriviaQAEvaluator.
No model card in ModelSpec reports this benchmark yet.