What it is
Synthetic benchmark for classifying why LLMs and AI agents fail, with root-cause labels and severity across realistic task domains. 1,500 synthetic prompt-context-response records across 5 domains, with expected vs. flawed answers, 12 root-cause failure classes, 4 severity levels, and rationale notes in a locked taxonomy.
Key results
- 1,500 rows and 10 columns
- 5 domains, 300 samples each
- 12 failure classes + 4 severity levels, Apache 2.0
Useful for training and benchmarking models that diagnose root-cause failures in LLM and agent outputs.