What it is
Confidence estimation from a model's own graded history: retrieve similar past episodes, read their hit rate, and revise stated confidence with one extra reflection. Instead of estimating confidence from only the current trace or resampling, it uses the model's own verified past episodes. It retrieves similar tasks with similar stated confidence, reads off historical success, then has the model restate confidence from that track record.
Key results
- AUROC beats or matches SC@10 on 23/24 model-benchmark comparisons
- Lower ECE than SC@10, with 3-8x lower error on MMLU-Pro
- Abstaining on the bottom 10% confidence raises agent success by up to 8.7 pts
A model's own verified experience is a stronger and cheaper basis for confidence than re-reading or resampling the current inference alone.