What it is
A 522-task image/video benchmark for intuitive visual reasoning with rubric-based grading. Humans score 93.1%, while the best tested model reaches 53.6%. 522 curated image/video QA tasks across 4 domains and 11 subdomains, each with a human question, reference answer, and 1-7 checkable rubric criteria.
Key results
- 522 tasks: 288 image and 234 video
- 4 domains and 11 subdomains
- 723 rubric criteria; MIT for annotations
Useful for evaluating multimodal models on intuitive visual reasoning beyond standard VQA.