What it is
A compact RAG evaluation benchmark with 800 queries, inlined retrieval, and 16 system answers per query for reference-free scoring across close- and open-ended QA. 800 English queries across 8 QA sources, split into close- and open-ended settings, with top-30 BM25 and ANCE chunks plus 16 answers per query from 8 RAG systems over both retrievers.
Key results
- 800 queries across 8 source datasets
- 2 retrievers: BM25 and ANCE, 30 chunks each
- 12,800 answers from 8 systems under both retrievers
Useful for benchmarking retrieval and answer quality in RAG with the same fixed queries and retrieved evidence.