Paper

Training Object Permanence in World Models

A cognitively grounded 3D video benchmark plus post-training setup that teaches object permanence and solidity through continuation tasks, not text labels or VLM scoring.

arXiv ↗Sep 23, 2026Code ↗

  • multimodal
  • video generation
  • world models
  • physical reasoning
Composite grid showing diverse physics simulation scenes the dataset contains
Composite grid showing diverse physics simulation scenes the dataset contains

What it is

Builds a 1.5M-sample benchmark and training corpus for object permanence/solidity in video world models, then fine-tunes a 16B continuation model on it. Instead of probing intuitive physics with small 2D benchmarks or VLM judges, it trains and evaluates video models on 150 hand-designed 3D continuation tasks where occlusion, containment, collision, and support events must be predicted.

Key results

  • 1.5M training samples from 150 Blender generators across 6 task families
  • PWM-WROP reaches Elo 1679.5, 1st among continuation models
  • 3rd overall of 14 models, behind only a tie between 2 reference-to-video systems

Shows that core-cognition-style synthetic data can measurably train stronger physical reasoning in video continuation models.

One useful ML email a week.

Worth your time, not all of your time.

We track new ML work across papers, code, and data, then keep only the pieces worth a closer look. Selected, not collected.

Papers.Data.Code

The Weekly ML Signal

Selected, not collected.

One useful ML email a week: the papers, repos and datasets worth your attention.

Worth your time, not all of your time.