V-FiLLM builds financial table-reasoning questions from executable computation trees so answers are correct by construction and difficulty is controllable.
unassessed
| Category | domain |
|---|---|
| Subcategory | compositional financial table reasoning |
| Page status | active |
| Metric | accuracy |
| Direction | higher_is_better |
| Unit | % |
| Publisher | ETH Zürich and Aisot Technologies Ltd |
V-FiLLM tests whether a language model can retrieve values from a financial table and compose arithmetic over them. Questions are rendered from typed expression trees whose leaves are spreadsheet cells (revenue, costs, assets, and named concepts such as gross profit). Difficulty is set independently along computation depth, expression breadth, financial-concept complexity, and context size. Optional multi-turn dialogues expose intermediate nodes, and adversarial table or wording perturbations test robustness without changing the gold answer.
English question over a synthetic 10-Q-style or regularized spreadsheet; the model returns a numeric answer scored against the tree-evaluated gold value, optionally across multi-turn sub-questions.
No model card in ModelSpec reports this benchmark yet.