HELM Enterprise yes/no classification of 11,412 gold-commodity headlines across nine information dimensions, scored by weighted F1.
unassessed
| Category | domain |
|---|---|
| Subcategory | HELM Enterprise yes/no classification of gold-commodity headlines |
| Page status | unknown |
| Metric | classification_weighted_f1 |
| Direction | higher_is_better |
| Dataset size | 11412 |
| Dataset licence | Data files © Original Authors |
| Publisher | Indian Institute of Management Ahmedabad (dataset); Stanford CRFM (HELM Enterprise scenario) |
gold_commodity_news is Stanford HELM's wrap of Sinha and Khandait's 2020 gold-headline set. Each item is one English news headline about gold. The model answers Yes or No for one of nine binary dimensions: whether the headline is about price, direction up, constant, or down, past or future price, past or future non-price news, or a comparison with another asset. HELM does not score a joint multi-label vector. The skill is financial headline tagging, not price forecasting.
Generation with get_generation_adapter_spec: instructions naming the dimension, input noun Headline, output noun Answer. Gold strings in the scenario are "Yes" or "No". Defaults: max_train_instances 5, max_tokens 5, temperature 0.0, stop at newline. Run spec name gold_commodity_news:category=<key>. Nine keys: price_or_not, direction_up, direction_constant, direction_down, past_price, future_price, past_news, future_news, assert_comparison.
No model card in ModelSpec reports this benchmark yet.