jfinqa tests multi-step numerical reasoning over real Japanese corporate financial statements from EDINET filings, across three subtasks: calculation, internal-consistency checking and trend direction.
unassessed
| Category | domain |
|---|---|
| Subcategory | numerical reasoning over Japanese corporate financial statements (EDINET filings): calculation, internal-consistency and year-over-year trend questions |
| Page status | active |
| Metric | exact_match and numerical_match (1% tolerance on numeric answers), reported overall and per subtask |
| Direction | higher_is_better |
| Unit | % |
| Dataset size | 1000 |
| Dataset licence | Apache-2.0 |
| Publisher | Independent project (GitHub user ajtgjmdjp); distributed simultaneously as a GitHub repository, a PyPI package and a Hugging Face dataset |
jfinqa tests whether a model can perform multi-step arithmetic over Japanese corporate financial statement tables pulled from real EDINET filings (the disclosure system run by Japan's Financial Services Agency), spanning J-GAAP, IFRS and US-GAAP accounting standards. It is not a classification or simple-lookup task: questions require one to six chained arithmetic steps -- growth-rate and margin calculations, ratio analysis, DuPont decomposition -- over a mix of pre-table text, a financial table, and post-table text, the same evidence shape FinQA (in this repository) uses for English SEC filings. Three separately-scored subtasks sit inside the one benchmark: Numerical Reasoning (calculate a financial metric), Consistency Checking (verify that reported figures agree with each other) and Temporal Reasoning (determine the direction of a year-over-year change).
Given pre-table text, a financial table and post-table text drawn from one company's EDINET filing, plus a question, the model generates a short free-text answer zero-shot -- a figure, a percentage, or a yes/no-style judgement depending on the subtask.
No model card in ModelSpec reports this benchmark yet.