What it is
Self-distillation for VLMs without labels or stronger teachers: the teacher sees the clean image, the student a degraded view, creating free on-policy supervision. Instead of privileged teachers or labels, it creates asymmetry by weakening the student view. An EMA teacher scores clean images while the student learns on augmented views, turning visual corruption into on-policy distillation signal.
Key results
- 77.44% avg, up from 70.68% on 6 perception benchmarks
- Beats Qwen3-VL-Instruct-235B: 77.44% vs 75.75% avg
- Recovers 96% of the gain from privileged-information methods
Teacher-student asymmetry for on-policy distillation can come from subtracting input information from the student, not adding privileged information to the teacher.